fsync()-in webdav PUT
valery+nginxen at grid.net.ru
Wed Feb 28 18:24:18 UTC 2018
It's completely clear why someone would need to flush file's data and
metadata upon a WebDAV PUT operation. That is because many architectures
expect a PUT operation to be completely settled before a reply is returned.
Without fsyncing file's data and metadata a client will receive a
positive reply before data has reached the storage, thus leaving
non-zero probability that states of two systems involved into a web
transaction end up inconsistent.
Further, the exact moment when the data of certain specific file reaches
the storage depends on numerous factors, for example, I/O contention.
Consequently, the exact moment when the data of a file being uploaded
reaches the storage can be only determined by executing fsync.
On 28-02-18 11:04, Aziz Rozyev wrote:
> While it’s not clear why one may need to flush the data on each http operation,
> I can imagine to what performance degradation that may lead of.
> if it’s not a some kind of funny clustering among nodes, I wouldn't care much
> where actual data is, RAM still should be much faster, than disk I/O.
>> On 28 Feb 2018, at 12:30, Nagy, Attila <bra at fsn.hu> wrote:
>> On 02/27/2018 02:24 PM, Maxim Dounin wrote:
>>>> Now, that nginx supports running threads, are there plans to convert at
>>>> least DAV PUTs into it's own thread(pool), so make it possible to do
>>>> non-blocking (from nginx's event loop PoV) fsync on the uploaded file?
>>> No, there are no such plans.
>>> (Also, trying to do fsync() might not be the best idea even in
>>> threads. A reliable server might be a better option.)
>> What do you mean by a reliable server?
>> I want to make sure when the HTTP operation returns, the file is on the disk, not just in a buffer waiting for an indefinite amount of time to be flushed.
>> This is what fsync is for.
>> Why doing this in a thread is not a good idea? It would'nt block nginx that way.
More information about the nginx