Skip to main content

#curl's configure script has 75(!) different --disable options to disable specific things in the build.

That's 37778931862957161709568 build combinations only there. Tricky to test.

#curl
in reply to daniel:// stenberg://

Kevin Lyda
Kevin Lyda
• •
Might that be a reason to reduce their number? To make the build more reasonable to test?
in reply to Kevin Lyda

daniel:// stenberg://
daniel:// stenberg://
• •
@lyda it's the constant battle and balance: providing the features and options people ask for and at the same time maintain quality and testability. So sure, I think every option should be questioned from that angle.
@Kevin Lyda
in reply to daniel:// stenberg://

Kevin Lyda
Kevin Lyda
• •

You maintain a 5,000+ line configure.ac file? Holy fuck. You poor man.

Folks, please help this guy get the therapy he needs:

https://opencollective.com/curl

#curl #ptsd #torment


curl - Open Collective

client-side internet transfers
opencollective.com
#curl #ptsd #torment
in reply to Kevin Lyda

daniel:// stenberg://
daniel:// stenberg://
• •

@lyda 5K is nothing 🤠

it is actually north of 17,000 lines counting all the individual additional .m4 files as well!

@Kevin Lyda
in reply to daniel:// stenberg://

Kevin Lyda
Kevin Lyda
• •
At this point do you actually understand m4? I admined sendmail back in the day and did a number of much smaller autoconf scripts for other projects back when I coded in C. But never really *got* m4.
in reply to Kevin Lyda

daniel:// stenberg://
daniel:// stenberg://
• •

@lyda it has been said that the 4 in m4 is the total amount of people understanding it, and I'm not one of them.

But I get by good enough pretending I know it. That's typically enough to survive and to write decently working configure stuff.

@Kevin Lyda
in reply to daniel:// stenberg://

Petr Tesarik
Petr Tesarik
• •
@lyda Oh no, and I hoped someone would finally explain dnl and changequote to me. Or at least the difference between [something] and [[something]].
@Kevin Lyda
in reply to daniel:// stenberg://

Daniel Janus
Daniel Janus
• •

An idea would be to randomly sample, say, 100 combinations out of that space using something like Vitter’s Method D [0], and test those.

I’ve been experimenting with this approach in my Clojure library [1].

[0]: https://www.ittc.ku.edu/~jsv/Papers/Vit87.RandomSampling.pdf

[1]: https://github.com/nathell/cartestian


GitHub - nathell/cartestian: Test all the combinations

Test all the combinations. Contribute to nathell/cartestian development by creating an account on GitHub.
GitHub
in reply to Daniel Janus

daniel:// stenberg://
daniel:// stenberg://
• •
@nathell I actually started out this morning writing a script that runs configure with a random collection of the 75 disable options. It revealed a lot already.
@Daniel Janus
in reply to daniel:// stenberg://

Ian Lewis :verified:
Ian Lewis :verified:
• •
If each test took a nanosecond to run, it would only take 1,197,962 years!
Unknown parent

daniel:// stenberg://
daniel:// stenberg://
• •
@berrange @lyda I don't think autoconf pushes for more options than anything else. It's just that these become build time options, not run-time as you would in many other places. Being able to switch off things at build time reduces attack surfaces that run-time checks cannot.
@Daniel P. Berrangé 📷 🎨 🔭 📡 @Kevin Lyda
in reply to daniel:// stenberg://

Fish Id Wardrobe
Fish Id Wardrobe
• •
that's … frightening. how do you decide which combinations to test? you don't have to answer
in reply to daniel:// stenberg://

vsz
vsz
• •
if we bring CMake to the party, it also has 61 enable/disable + 30 dependency flags. That's 91 bits of combinations right there. Throw in some more with platform, compiler and compiler versions that all trigger a slightly different compilation path.
This entry was edited (1 month ago)
in reply to vsz

daniel:// stenberg://
daniel:// stenberg://
• •
@vsz most of these combo problems are in the source code though, independent of cmake vs autotools
@vsz
in reply to daniel:// stenberg://

vsz
vsz
• •
Indeed. It me made thinking to do a translation table between configure and cmake options to spot remaining differences.
This entry was edited (1 month ago)
in reply to daniel:// stenberg://

awb
awb
• •
This problem is what Combinatorial Interaction Testing is looking to address. This 2013 IEEE Computer article on testing the 1.8×10^55 combinations of the Apache config is a good description of that approach (https://doi.org/10.1109/MC.2013.408).

Moving Forward with Combinatorial Interaction Testing

Combinatorial interaction testing (CIT) is an effective failure detection method for many types of software systems.
doi.org
in reply to daniel:// stenberg://

Klaus Frank
Klaus Frank
• •

Ngl, that's one of the main reasons I hate C/C++ projects. It is basically impossible to just change one line or two to test something without first spending at least a week or two getting into all of the project despite already knowing which line of code you actually want to change.

Followed by not having any kind of dependency management...

⇧