{"id":803,"date":"2019-04-19T19:39:39","date_gmt":"2019-04-20T02:39:39","guid":{"rendered":"http:\/\/www.earlevel.com\/main\/?p=803"},"modified":"2023-02-07T03:38:28","modified_gmt":"2023-02-07T11:38:28","slug":"floating-point-denormals","status":"publish","type":"post","link":"https:\/\/www.earlevel.com\/main\/2019\/04\/19\/floating-point-denormals\/","title":{"rendered":"Floating point denormals"},"content":{"rendered":"

There\u2019s another issue with floating point hardware that can easily cause serious performance problems in DSP code. Fortunately, it\u2019s also easy to guard against if you understand the issue. I covered this topic a few years ago in A note about de-normalization<\/a>, but giving it a fresh visit as a companion to Floating point caveats<\/a>.<\/p>\n

Floating point hardware is optimized for maximum use of the mantissa, for best accuracy. (It also allows for more efficient math processing, since the mantissas are always aligned.) This is called normalization\u2014all numbers are kept in the binary range of plus or minus 1.1111111 (keep going, 16 more 1s) times 2 raised to the power of the exponent. To increase accuracy near zero, floating point implementations let the number become \u201cdenormalized\u201d, so instead of the smallest number being 1.0 times 2 raised to the most negative exponent, the mantissa can become as small as 0.000\u20261 (24 digits).<\/p>\n

The penalty is that floating point math operations become considerably slower. The penalty depends on the processor, but certainly CPU use can grow significantly\u2014in older processors, a modest DSP algorithm using denormals could completely lock up a computer.<\/p>\n

But these extremely tiny values are of no use in audio, so we can avoid using them, right? Not so easily\u2014recursive algorithms such as IIR lowpass filters can decay into near-zero numbers in typical use, so they will happen. For instance, when you hit Stop on your DAW\u2019s transport, it likely sends a stream of zeros (to let reverbs decay out, and any live input through effects to continue). The memory of a lowpass filter then decays exponentially towards zero. We can slow it a bit by using double precision floats, but it\u2019s only a matter of time till the processing meter climbs abruptly when your processing algorithm bogs down in denormalized computation.<\/p>\n

Modern processors can mitigate the problem with flush-to-zero and denormals-are-zero modes. But this too can be tricky\u2014you\u2019re sharing the processor with other functions that might fail without the expected denormal behavior. You could flip the switch off and back during your main DSP routine, but you need to be careful that you\u2019re not calling a math function that might fail. Also, this fix is processor dependent, and might be the wrong thing to do if you move your code ot a new platform. A DSP framework might handle this conveniently for you, but my goal here is to show you that it\u2019s pretty easy to work around, even without help from the processor. Let\u2019s look at what else we could do.<\/p>\n

We could test the output of any susceptible math operation\u2014there is no need to test all operations, because most won\u2019t create a denormal without a denormal as input.<\/p>\n

There\u2019s a surprisingly easy solution, though, and it\u2019s more efficient than testing and handling each susceptible operation. At points that have the threat of falling into denormals, we can add a tiny value to ensure it can\u2019t get near zero. This value can be so small that it\u2019s hundreds of dB down from the audio level, but it\u2019s still large compared with a denormal.<\/p>\n

In fact, it\u2019s actually possible to flush denormals to zero with no other error, due to the same properties of floating point we discussed earlier. If you add a relatively large number to a denormal, then subtract it back out, the result is zero. Still, this is pointless, because a tiny offset will not be heard.<\/p>\n

But there is a catch to watch out for. A constant value in the signal path is the same as a DC offset. Some components, such as a highpass filter, remove DC offsets. I get around this by using a tiny number to add as need in the signal path\u2014often, one time is enough for an entire plugin channel\u2014and alternating the sign of it each time a new buffer is processed. A value such as 1e-15 (-300 dB) completely wipes out denormals while having no audible effect.<\/p>\n

Why did I pick that value? It needs to be at least the smallest normalized number (2^-126, about 1e-38, or about -760 dB), to ensure it wipes out the smallest denormal. That would work, but it\u2019s better to use a larger number so that you don\u2019t have to add it in as often. A much larger number would likely be fine to add just once\u2014just add it to the input sample, for instance. (But consider your algorithm\u2014if you have a noise gate internal to your effect, you might need to add the the \u201cfixer\u201d value after its output as well.) And clearly it needs to be small enough that it won\u2019t be heard, and won\u2019t affect algorithms such as spectrum analysis. A number like 1e-15 is -300 dB\u2014each additional decimal place is -20 dB, so 1e-20 is -400 dB, for instance. Let your own degree of paranoia dictate the value, but numbers in those ranges are unlikely to affect DSP algorithms, while wiping out any chance of denormals through the chain.<\/p>\n

To recap: Take a small number (mDenormGuard = 1e-15;). Change its sign (mDenormGuard = -mDenormGuard;) once in a while so a DC blocker you might have in your code doesn\u2019t remove it. A handy place is the beginning of your audio buffer handling routine, one sign change for an entire buffer should be fine. Add it in (input + mDenormGuard).<\/p>\n

Here’s an example of my plug-in (Amp Farm native), during development, showing the performance meter in a DAW while processing audio:<\/p>\n

\"\"<\/a><\/p>\n

With denormal protection turned off, the performance is the same while processing normal audio, but a few seconds after stopping the transport, the load rises due to denormals:<\/p>\n

\"\"<\/a><\/p>\n

With denormal protection enabled, performance looks like the first picture at all times. And that’s why we protect against denormals!<\/p>\n","protected":false},"excerpt":{"rendered":"

There\u2019s another issue with floating point hardware that can easily cause serious performance problems in DSP code. Fortunately, it\u2019s also easy to guard against if you understand the issue. I covered this topic a few years ago in A note … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[38],"tags":[],"_links":{"self":[{"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/posts\/803"}],"collection":[{"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/comments?post=803"}],"version-history":[{"count":14,"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/posts\/803\/revisions"}],"predecessor-version":[{"id":1356,"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/posts\/803\/revisions\/1356"}],"wp:attachment":[{"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/media?parent=803"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/categories?post=803"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.earlevel.com\/main\/wp-json\/wp\/v2\/tags?post=803"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}