It is not possible to create a case insensitive index in mongodb yet. Because of this, our case-insensitive searches are collections scans, and it is slow.
A possible hack to deal with this is to add case insensitive flag to metadata index definitions, and if a field has a case insensitive index, have mongo back end create a new hidden field with the lower case value of the original field. We can rewrite queries for fields in case-insensitive indexes to use the hidden field. As an example, if an entity has a case insensitive index on 'field1', all documents would be saved with: ... field1: "field1Value", field1__lc: "field1value" and case insensitive regex queries using "field1" will be translated to use "field1__lc". field1__lc (we can find a better naming scheme) won't be translated for retrieved documents, so this scheme is transparent and safe except for direct mongo updates. |
Administrator
|
This sounds reasonable to me. On Thu, Jan 7, 2016 at 12:48 AM bserdar [via lightblue-dev] <[hidden email]> wrote: It is not possible to create a case insensitive index in mongodb yet. Because of this, our case-insensitive searches are collections scans, and it is slow. |
Index keys now have a caseInsensitive flag:
"indexes": [ { "unique":true, "fields": [ {"field":"field1", "dir":"$asc", "caseInsensitive":"true" }, {"field":"field2", "dir":"$asc" } ] }, { "unique":true, "fields": [ {"field":"field3", "dir":"$asc", "caseInsensitive":"true" } ] } ] The difference in how this is handled between lightblue and mongo is hidden from the user, but important. The above caseInsensitive indexes will actually create indexes on the following fields: `@mongoHidden.field1` `@mongoHidden.field3` And in turn, updates to the caseInsensitive fields will also update corresponding hidden fields: { "field1": "testField1", "field2": "testField2", "field3": "testField3", "@mongoHidden.field1": "TESTFIELD1", "@mongoHidden.field3": "TESTFIELD3" } This is all entirely hidden from the user, and they will only be privy to the fields they've actually defined in metadata. |
Administrator
|
Is it possible to get access to these by creating a field in lightblue metadata that matches the upper case field name? On Tue, Feb 2, 2016 at 1:24 PM bvulaj [via lightblue-dev] <[hidden email]> wrote: Index keys now have a caseInsensitive flag: |
This case insensitive index implementation is mongo specific. Another
backend would implement it differently without hidden fields. So it wouldn't be a good idea to expose those. We can add a feature to define fields as upper/lower case. On Mon, Feb 8, 2016 at 5:15 PM, jewzaam [via lightblue-dev] <[hidden email]> wrote: > Is it possible to get access to these by creating a field in lightblue > metadata that matches the upper case field name? > > On Tue, Feb 2, 2016 at 1:24 PM bvulaj [via lightblue-dev] <[hidden email]> > wrote: >> >> Index keys now have a caseInsensitive flag: >> >> "indexes": [ >> { >> "unique":true, >> "fields": [ >> {"field":"field1", >> "dir":"$asc", >> "caseInsensitive":"true" >> }, >> {"field":"field2", >> "dir":"$asc" >> } >> ] >> }, >> { >> "unique":true, >> "fields": [ >> {"field":"field3", >> "dir":"$asc", >> "caseInsensitive":"true" >> } >> ] >> } >> ] >> >> The difference in how this is handled between lightblue and mongo is >> hidden from the user, but important. The above caseInsensitive indexes will >> actually create indexes on the following fields: >> >> `@mongoHidden.field1` >> `@mongoHidden.field3` >> >> And in turn, updates to the caseInsensitive fields will also update >> corresponding hidden fields: >> >> { >> "field1": "testField1", >> "field2": "testField2", >> "field3": "testField3", >> "@mongoHidden.field1": "TESTFIELD1", >> "@mongoHidden.field3": "TESTFIELD3" >> } >> >> This is all entirely hidden from the user, and they will only be privy to >> the fields they've actually defined in metadata. >> >> ________________________________ >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://dev.forum.lightblue.io/Case-insensitive-search-support-tp409p412.html >> To start a new topic under lightblue-dev, email [hidden email] >> To unsubscribe from lightblue-dev, click here. >> NAML > > > > ________________________________ > If you reply to this email, your message will be added to the discussion > below: > http://dev.forum.lightblue.io/Case-insensitive-search-support-tp409p413.html > To start a new topic under lightblue-dev, email > [hidden email] > To unsubscribe from lightblue-dev, click here. > NAML |
In reply to this post by jewzaam
There is a check to make sure that metadata doesn't contain any @mongoHidden prefixed fields. @mongoHidden is now effectively a reserve word.
So, no, even if they wanted to, a user would not be able to expose these fields. |
Free forum by Nabble | Edit this page |