<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="308496" data-post-id="308496">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="kip" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/kip/120/1440_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  kip
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>ex_cldr Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>You can get quite a long way just using the Elixir lexer. Especially now that in Elixir 1.16 its lexing errors are better for user experience.</p>
<h3><a name="p-308496-example-1" class="anchor" href="#p-308496-example-1" aria-label="Heading link" rel="nofollow"></a>Example</h3>
<pre data-code-wrap="elixir"><code class="lang-elixir">iex&gt; Code.string_to_quoted "(\"Modesty Blase\" or \"Willie Garvin\")"
{:ok, {:or, [line: 1], ["Modesty Blase", "Willie Garvin"]}}

iex&gt; Code.string_to_quoted "a or (b and c)"
{:ok,
 {:or, [line: 1],
  [
    {:a, [line: 1], nil},
    {:and, [line: 1], [{:b, [line: 1], nil}, {:c, [line: 1], nil}]}
  ]}}

iex&gt; Code.string_to_quoted "a or (b and c"
{:error,
 {[opening_delimiter: :"(", line: 1, column: 14],
  "missing terminator: ) (for \"(\" starting at line 1)", ""}}
</code></pre>
<p>You would then need a parser to ensure you have a valid expression for your query language but I think thats really quite easy compared to the lexical part given your reasonably straight forward requirements.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="308496" data-batch-url="/posts/batch_likers">
                        5
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/12">Post #11</a>
	                </div>
	            </div>
              <div id="likers-container-308496" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="308496"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #11"></div>
  </section>
</div>
    <div class="postbit" id="309232" data-post-id="309232">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="elvanja" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/elvanja/120/17393_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  elvanja
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Hmmm, interesting idea! Unfortunately, it does not solve certain edge cases out of the box, e.g.:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">iex&gt; Code.string_to_quoted(~s{"Modesty Blaze" AND "Willie Garvin"})
{:error, {[line: 1, column: 17], "syntax error before: ", "'AND'"}}
</code></pre>
<p>Didn’t have time to inspect further since I ended up with a solution that just scans the input, see next answers.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309232" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/13">Post #12</a>
	                </div>
	            </div>
              <div id="likers-container-309232" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309232"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #12"></div>
  </section>
</div>
    <div class="postbit" id="309234" data-post-id="309234">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="elvanja" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/elvanja/120/17393_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  elvanja
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>This idea to just implement binary pattern matching got me thinking. I couldn’t find a way to actually do that. But, during the study of the issue, decided to go with simple scan approach. E.g. just go over each character, split on spaces and count parenthesis and quotes. I have the working solution at <a href="https://gist.github.com/elvanja/202b31ae5735f297956e1d8f567c7d7e" class="inline-onebox" rel="noopener nofollow ugc">Boolean query lexer (not AST unfortunately) · GitHub</a> if you care to take a look. Not simple though and likely hard to debug too. But this bought time so I can try and implement this via some other mechanism.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309234" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/14">Post #13</a>
	                </div>
	            </div>
              <div id="likers-container-309234" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309234"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #13"></div>
  </section>
</div>
    <div class="postbit" id="309236" data-post-id="309236">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="elvanja" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/elvanja/120/17393_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  elvanja
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Good point about silently ignoring grammar errors! Haven’t considered that one and it does make sense. E.g. given search for <code>Modesty Blase OR Willie Garvin AND</code> it makes sense to drop the last <code>AND</code> since it does not serve any purpose.</p>
<p>Would love to formalize requirements but unfortunately this will go the “let’s see how it works and then we’ll improve” way. Not that this is bad, just that it does not yield to formalization well <img src="https://forum.elixirforum.com/images/emoji/apple/smile.png?v=15" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
<p>So for the time being I used a plain approach, scanning the input. See the gist link in <a href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/14" rel="nofollow">reply above</a>. This solution does that, ignores certain input errors (drops tailing operators, uses only the last operator if more than one are supplied, …). It does not say at which character e.g. unclosed parenthesis started, but it should be fairly simple to add that too (it is scanning so counting should not be an issue).</p>
<p>I do plan to try some/all of those solutions and see if I can get a more maintainable solution after all. Hope it will be possible to also ignore certain “invalid” input and have nice enough error reporting.</p>
<p>Thanks for the idea! <img src="https://forum.elixirforum.com/images/emoji/apple/bowing_man.png?v=15" title=":bowing_man:" class="emoji" alt=":bowing_man:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309236" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/15">Post #14</a>
	                </div>
	            </div>
              <div id="likers-container-309236" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309236"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #14"></div>
  </section>
</div>
    <div class="postbit" id="309266" data-post-id="309266">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Adzz" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Adzz/120/10743_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Adzz
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>That’s because the AND is uppercased, might be a simple fix with a String.replace ?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309266" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/16">Post #15</a>
	                </div>
	            </div>
              <div id="likers-container-309266" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309266"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #15"></div>
  </section>
</div>
    <div class="postbit" id="309447" data-post-id="309447">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="elvanja" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/elvanja/120/17393_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  elvanja
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Yep, replacing would work for operators. Mind you, this is user input so we could expect also things like <code>aNd</code>, <code>And</code> etc. and need to support another language too. But it could be done.</p>
<p>Unfortunately, apart from quoted terms, it needs to support non-quoted multi-word ones, e.g.:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">iex(1)&gt; Code.string_to_quoted(~s{SAP ABAP})
{:error, {[line: 1, column: 5], "syntax error before: ", "'ABAP'"}}
</code></pre>
<p>This case breaks it because those terms are indeed user input and not part of any dictionary, so have to take it in and parse as-is (no transformation/downcasing).</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309447" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/17">Post #16</a>
	                </div>
	            </div>
              <div id="likers-container-309447" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309447"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #16"></div>
  </section>
</div>
    <div class="postbit" id="309448" data-post-id="309448">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="akash-akya" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/akash-akya/120/18688_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  akash-akya
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>If strings can be unquoted then it will become ambiguous grammer right? For example how should <code>FOO AND BAR</code> be parsed? It can be both a boolean expression and a single multi word string.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309448" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/18">Post #17</a>
	                </div>
	            </div>
              <div id="likers-container-309448" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309448"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #17"></div>
  </section>
</div>
    <div class="postbit" id="309875" data-post-id="309875">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="th3mus1cman" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/th3mus1cman/120/32791_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  th3mus1cman
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p><a class="mention" href="/u/elvanja" rel="nofollow">@elvanja</a> You can take a look at what I have done here <a href="https://github.com/themusicman/predicated/blob/main/lib/predicated/query/parser.ex" class="inline-onebox" rel="noopener nofollow ugc">predicated/lib/predicated/query/parser.ex at main · themusicman/predicated · GitHub</a>.</p>
<p>Here are some examples in the limited tests: <a href="https://github.com/themusicman/predicated/blob/main/test/predicated/query/parser_test.exs" class="inline-onebox" rel="noopener nofollow ugc">predicated/test/predicated/query/parser_test.exs at main · themusicman/predicated · GitHub</a></p>
<p>I am stuck at the moment on supporting parans and nesting.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309875" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/19">Post #18</a>
	                </div>
	            </div>
              <div id="likers-container-309875" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309875"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #18"></div>
  </section>
</div>
    <div class="postbit" id="309993" data-post-id="309993">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="th3mus1cman" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/th3mus1cman/120/32791_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  th3mus1cman
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Well, I had a break through last night and added support for grouping and nested grouping. Might be some bugs in there but I think it generally works.</p>
<p><a href="https://github.com/themusicman/predicated/commit/cca03c4e15206cdb7d1bf409f4d9c322b8106c29#diff-2291a21e493bd663ef36b5e16bfdf8a6a09ae25a2fa5d26dbed6a970969db52cR1" class="onebox" target="_blank" rel="noopener nofollow ugc">https://github.com/themusicman/predicated/commit/cca03c4e15206cdb7d1bf409f4d9c322b8106c29#diff-2291a21e493bd663ef36b5e16bfdf8a6a09ae25a2fa5d26dbed6a970969db52cR1</a></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="309993" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/20">Post #19</a>
	                </div>
	            </div>
              <div id="likers-container-309993" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="309993"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #19"></div>
  </section>
</div>
    <div class="postbit" id="311307" data-post-id="311307">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="elvanja" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/elvanja/120/17393_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  elvanja
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Well, we can make some assumptions here, I think. E.g. if user would enter “SAP BW OR Java” in search engine I would assume the user wanted someone with “SAP BW” <strong>OR</strong> “Java”. The way I decided to interpret unquoted searches basically boils down to considering everything between boolean operators (AND, OR, NOT, AND NOT, …) to be a “keyword” term which can be a single or multi word in nature. One needs to be careful with quoted terms of course (e.g. to take <code>"Bang AND Olufsen"</code> - quoted - as is).</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="311307" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/recommendation-for-building-search-parser/59807/21">Post #20</a>
	                </div>
	            </div>
              <div id="likers-container-311307" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="311307"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #20"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <a class="load-more-button" data-turbo-stream="true" href="/topics/59807/load_more?page=3">Load more posts (1 remaining)</a>
</div></template></turbo-stream>