On test-suite coverage

[Hugo Lefeuvre]

Hi Maxim,

On Fri, Jan 14, 2022 at 07:33:20PM +0300, Maxim Konovalov wrote:
> This is more or less in line with our numbers.  Just from today report for
> mainline: 77.8% for lines coverage and 93.1% for functions.  These figures
> include njs code though.
> 
> > I was wondering if I did something wrong in my measurements, if this is a
> > known weakness of the test-suite, and in the latter case, if this is
> > something that the Nginx project is open to receiving contributions on.
> 
> Probably not a weakness but unjustified expectations? :-)
> 
> I think the biggest non-covered part comes from various errors paths.
> Some of them could be hard to trigger.  For example, for memory
> allocation failures we use separate nodes with modified ngx_palloc.c
> which enables random memory allocation errors.  This is not a part of
> the standard test suite though.

Thanks for confirming the numbers! Makes sense. We made the same
observation with error paths, and it's nice to confirm it. I suppose that
stimulating some of these error paths would require approaches (syscall
interception, some degree of mocking) that might be in contradiction with
the requirements you listed below (compactness, CI/CD).

> Anyway, any meaningful contributions in this area will be highly
> appreciated.
> 
> http://nginx.org/en/docs/contributing_changes.html
> 
> There are several things to keep in mind: we want to keep the suite compact
> as it is now and be able to have it integrated into CI/CD systems easily.

Alright, we will definitely attempt to upstream any test that we might have
to write for this project. :-)

Best,
Hugo

-- 
                Hugo Lefeuvre (hle)    |    www.owl.eu.com
RSA4096_ 360B 03B3 BF27 4F4D 7A3F D5E8 14AA 1EB8 A247 3DFD
ed25519_ 37B2 6D38 0B25 B8A2 6B9F 3A65 A36F 5357 5F2D DC4C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220115/035c7eda/attachment.bin>