search sparse field

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

search sparse field

Leonardo Pistone
Hi all,

Has anyone worked on a search() method for JSON-serialized sparse fields?

In theory, one could use postgresql's JSON features introduced in 9.2, or failing that, some (dirty) regexp to build the search query.

Any ideas or suggestions?

I've seen Therp's unserialize_field, but I like to stay serialized :)

Thanks!

_______________________________________________
Mailing list: https://launchpad.net/~openerp-expert-framework
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-expert-framework
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: search sparse field

Raphael Valyi
On Tue, Dec 10, 2013 at 10:54 AM, Leonardo Pistone <[hidden email]> wrote:
Hi all,

Has anyone worked on a search() method for JSON-serialized sparse fields?

In theory, one could use postgresql's JSON features introduced in 9.2, or failing that, some (dirty) regexp to build the search query.

Any ideas or suggestions?

I've seen Therp's unserialize_field, but I like to stay serialized :)

Thanks!


Hello Leonardo,

For flexibility I would favor evolving the sparse field feature from a text field to a native PostgreSQL json field. I mean I say this because I think it's better than moving to hstore and getting the search feature from hstore.
I have some experience extending custom_product_attributes into a product configurator (though funding was dropped 2 years ago) and I can tell you nesting is definitely a must have in the big picture.

Then yes, I would naively go for building a regexp to add some search support upon a full text indexed field.

Also notice that if you admit a heavier solution, I built an Apache SolR connector providing much more advanced search features https://github.com/akretion/solerp/tree/master/solerp
The real case I've been working has nearly 1 millions of products indexed into Apache SolR, and it's indexing thousands of attributes that use the product_custom_attributes and hence sparse fields NOSQL power.

I'm curious what other think about this too.

Regards.

-- 
Raphaël Valyi
Founder and consultant
+55 21 2516 2954



_______________________________________________
Mailing list: https://launchpad.net/~openerp-expert-framework
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-expert-framework
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: search sparse field

Leonardo Pistone
Thanks for yor answer Raphael.

On the long run, I agree with you that using a native Postgresql field seems like the obvious choice.

For now, searching in the text-based json fields seems something to implement with a Regexp. Otherwise, maybe with postgresql 9.2 we can use the native json methods to search in the text field, but I'm not sure.

Best, Leo


On Tue, Dec 10, 2013 at 2:07 PM, Raphael Valyi <[hidden email]> wrote:
On Tue, Dec 10, 2013 at 10:54 AM, Leonardo Pistone <[hidden email]> wrote:
Hi all,

Has anyone worked on a search() method for JSON-serialized sparse fields?

In theory, one could use postgresql's JSON features introduced in 9.2, or failing that, some (dirty) regexp to build the search query.

Any ideas or suggestions?

I've seen Therp's unserialize_field, but I like to stay serialized :)

Thanks!


Hello Leonardo,

For flexibility I would favor evolving the sparse field feature from a text field to a native PostgreSQL json field. I mean I say this because I think it's better than moving to hstore and getting the search feature from hstore.
I have some experience extending custom_product_attributes into a product configurator (though funding was dropped 2 years ago) and I can tell you nesting is definitely a must have in the big picture.

Then yes, I would naively go for building a regexp to add some search support upon a full text indexed field.

Also notice that if you admit a heavier solution, I built an Apache SolR connector providing much more advanced search features https://github.com/akretion/solerp/tree/master/solerp
The real case I've been working has nearly 1 millions of products indexed into Apache SolR, and it's indexing thousands of attributes that use the product_custom_attributes and hence sparse fields NOSQL power.

I'm curious what other think about this too.

Regards.

-- 
Raphaël Valyi
Founder and consultant
<a href="tel:%2B55%2021%202516%202954" value="+552125162954" target="_blank">+55 21 2516 2954




_______________________________________________
Mailing list: https://launchpad.net/~openerp-expert-framework
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-expert-framework
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: search sparse field

Leonardo Pistone


On Tue, Dec 10, 2013 at 2:18 PM, Leonardo Pistone <[hidden email]> wrote:
Thanks for yor answer Raphael.

On the long run, I agree with you that using a native Postgresql field seems like the obvious choice.

For now, searching in the text-based json fields seems something to implement with a Regexp. Otherwise, maybe with postgresql 9.2 we can use the native json methods to search in the text field, but I'm not sure.

Best, Leo


On Tue, Dec 10, 2013 at 2:07 PM, Raphael Valyi <[hidden email]> wrote:
On Tue, Dec 10, 2013 at 10:54 AM, Leonardo Pistone <[hidden email]> wrote:
Hi all,

Has anyone worked on a search() method for JSON-serialized sparse fields?

In theory, one could use postgresql's JSON features introduced in 9.2, or failing that, some (dirty) regexp to build the search query.

Any ideas or suggestions?

I've seen Therp's unserialize_field, but I like to stay serialized :)

Thanks!


Hello Leonardo,

For flexibility I would favor evolving the sparse field feature from a text field to a native PostgreSQL json field. I mean I say this because I think it's better than moving to hstore and getting the search feature from hstore.
I have some experience extending custom_product_attributes into a product configurator (though funding was dropped 2 years ago) and I can tell you nesting is definitely a must have in the big picture.

Then yes, I would naively go for building a regexp to add some search support upon a full text indexed field.

Also notice that if you admit a heavier solution, I built an Apache SolR connector providing much more advanced search features https://github.com/akretion/solerp/tree/master/solerp
The real case I've been working has nearly 1 millions of products indexed into Apache SolR, and it's indexing thousands of attributes that use the product_custom_attributes and hence sparse fields NOSQL power.

I'm curious what other think about this too.

Regards.

-- 
Raphaël Valyi
Founder and consultant
<a href="tel:%2B55%2021%202516%202954" value="+552125162954" target="_blank">+55 21 2516 2954





_______________________________________________
Mailing list: https://launchpad.net/~openerp-expert-framework
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-expert-framework
More help   : https://help.launchpad.net/ListHelp
Loading...