A hardware question

[mike]

I think it would be something worthwhile myself.

Just needs parameters - the tracker database info, and the domain
right? Then you just use the domain + key to get the file. Maybe
domain isn't even required.

However, wouldn't you need some sort of logic to determine how to
create the key?

Some people use an md5 of the URI I guess, so it would be something like

foo.com/media/abc72849482bc8398af98effd9g843

/media/ would be mapped to something like

option #1 would be nginx do the mysql connection itself.
option #2 would be nginx just connects to the tracker (this is
probably best) since it's a simple socket connection

location /media/ {
mogilefs on;
mogilefs_domain images;

option #1
mogilefs_tracker_username mogilefs;
mogilefs_tracker_password foo;
mogilefs_tracker_host 192.168.1.203;
mogilefs_tracker_database mogilefs;

option #2
mogilefs_tracker_host 192.168.1.202:7001;

and perhaps something like mogilefs_key_style hash; ? or something...
(used to determine how the keys are actually setup)
}

I think the biggest missing piece of the puzzle is how to determine
the key structure. Otherwise I am sure it's quite simple, and it uses
dav/HTTP to retrieve the files anyway, so the C code needed would be:

option #1:
1) mysql client
2) something which can do efficient HTTP GETs with offset support
(probably already in nginx)

option #2:
1) something that can do a simple socket connect, ask the tracker for
info, and then pass it on to step 2 (probably already in nginx)
2) something which can do efficient HTTP GETs with offset support
(probably already in nginx)

If I knew anything worthwhile in C I would try it myself. I am sure
someone even with basic C skills could hack it up really quick. It
just needs one socket to talk to the tracker, then depending on the
reply, open a second connection to grab the file...


On 4/28/08, kingler <kingler at 72pines.com> wrote:
> I know there are MogileFS client libraries for Perl/PHP/Ruby, but if
> someone can code a C library for MogileFS and it is then possible to
> make a Nginx module to support. I guess this will be more efficient
> than going down the Perl/PHP/Ruby route.
>
> Does it make sense at all? Would the benefits outweigh the extra
> efforts to code the nginx module?
>
> -Liang
>
> kingler from 72pines
>
> On Mon, Apr 28, 2008 at 11:46 AM, mike <mike503 at gmail.com> wrote:
> > I am actually thinking about making a MogileFS passthrough
> >
> >  (I posted an email about it, looks like someone in Japan figured it out)
> >
> >  Would allow for minimal PHP/perl/python/whatever language to be
> >  involved, just enough to look up where a file is. Then pass that back
> >  to nginx with "file is on $server" and then use X-Accel-Redirect to
> >  bounce to that server...
> >
> >  Haven't done it yet, but I think it would work quite amazing!
> >
> >
> >
> >  On 4/28/08, Igor Sysoev <is at rambler-co.ru> wrote:
> >  > On Mon, Apr 28, 2008 at 12:16:59PM +0200, Chavelle Vincent wrote:
> >  >
> >  > > Igor Sysoev wrote:
> >  > >
> >  > > > We use the proxying in this case instead of NFS:
> >  > > >
> >  > > > client   >      nginx (1)   >      nginx
> >  > > >
> >  > > > On nginx (1) it's better to set "proxy_max_temp_file_size 0" for
> >  > > > the proxied location.
> >  > >
> >  > > You recommend to use proxying for mass file serving. I am in this case,
> >  > > I use nginx and a specific module that I have coded. (to simplify
> >  > > things, it control access files and redirect client to
> >  > > ngx_http_internal_redirect)
> >  > >
> >  > > I have think to use NFS for serve the files. I don't understand how to
> >  > > use proxying instead of that.
> >  >
> >  > Set up nginx on NFS server and proxy to it instead of getting files via NFS:
> >  >
> >  >        location /files/ {
> >  >             proxy_pass                 http://nfs_server_nginx;
> >  >             proxy_max_temp_file_size   0;
> >  >        }
> >  >
> >  >
> >  > --
> >  > Igor Sysoev
> >  > http://sysoev.ru/en/
> >  >
> >  >
> >
> >
>
>