Architecture for asynchronous Mongo GridFS module for Nginx

[Tigran Bayburtsyan]

Hi , I’ve tried to do same thing, but new https://github.com/mongodb/mongo-c-driver  is extremely buggy and still don’t have support for async cursor reading or keep alive connection (it keeps only handler structure for making new connection on the fly). 

After a week of development and researches I got some working version but it’s didn’t work out on production, on existing database new mongo-c driver tried to make new kind of indexes which is not good for our database. 

So at the end I’ve just created new concurrent service with Go and upstream for nginx …. And it works perfectly !!

 

I’ve tried to change Mongo C driver to work with Nginx upstream API, but I ended up with a lot of big changes, so I just gave up.

I think using separate service for GridFS is more scalable than making it inside Nginx, because you can just make upstream to 100s of GrdiFS frontend services like I did.

 

But anyway if you will do something like that I’ll defiantly contribute !!

 

Thanks 

 

From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Alexander Rodin
Sent: Monday, January 18, 2016 7:04 PM
To: nginx-devel at nginx.org
Subject: Architecture for asynchronous Mongo GridFS module for Nginx

 

Hi all!

I'm researching possible approaches to development of asynchronous GridFS module for Nginx (because the existing one blocks the main Nginx thread).

I've read the source codes of most modules from https://www.nginx.com/resources/wiki/modules/ and found that all of asynchronous modules (e.g. Postgres or Mongo) implement the database protocol by itself working with upstream. Is it the only way? I would like to use mongo-c-driver (https://github.com/mongodb/mongo-c-driver) but I need an example how to embed it into Nginx event loop to make non-blocking module.

Could you suggest any module that uses the same approach or some manual about it?

Best regards

Alexander

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160118/e2932c0c/attachment.html>