How we test for reliability
Fuzz tests, memory checkers, plug-in validators and failure cases: what we test, how often and what came out. No promise that nothing ever happens, but numbers with dates.

We show how we look for problems and what came out of it. The numbers come from our test logs.
Damaged and malicious files: for the formats you can load into PSOrvia there are fuzz tests. A program builds millions of twisted variants from valid sample files and throws them at the reader. There are more than 80 such test targets: audio formats, presets and settings, fonts, images, PDF, playlists, podcast feeds, radio track information, server replies and network protocols. In total more than 180 million inputs went through. With them we found more than 10 defects, for example an endless loop caused by a single null byte in a preset file and a memory overflow on a file with a too short header field. All are fixed, and every finding stays as a test file that is replayed first before each new run.
Memory and undefined behaviour: the tests also run under checkers that report every memory access and every undefined calculation. On Linux these checks run without errors: 413 checks of the radio streams, 70 of the WebDAV client, 78 of the OSC control and 38 for HTTP and HTTPS.
Plug-in validators: our plug-in round consists of a strict validator (strictness 8 of at most 10) that changes parameters at random, switches channel layouts and saves and reloads the state, plus one checker each for VST3 and CLAP and, on macOS, the Audio Unit check. In a sample of 5 PSOrvia plug-ins, all 5 passed the strict validator.
Automatic host tests: a test program loads plug-ins without a screen into a real music program, reads the parameters, sets one and checks that it stays, and renders 2 seconds that must be neither missing nor silent. Currently 3 of 3 cases pass (one plug-in in three formats).
When something does go wrong: a crashed plug-in in its own guest process does not take the other tracks with it; the page "A crashed plug-in takes nothing with it" describes that. That the sound does not change unnoticed is checked by the page "Checked bit by bit". Here we add the network side: radio streams reconnect after a break with a growing pause and were tested with cut and stalled streams and damaged track information. A network share that cannot be reached is a state, not an error: PSOrvia Player tries again with growing pauses (5, 15, 30, 60 seconds) and the share comes back by itself. If a program crashes, the Player writes a report to a local file; nothing is sent.