fix float precision bug in text box rendering

[unterkoefler]

fixes #1292

[pointlessone]

Could you please add a test that demonstrates validity of this fix?

Also could you please explain the value of that number? Why can't it be smaller/bigger?

[unterkoefler]

This commit adds a test that fails, but it does so by manually inserting the necessary rounding errors, so I didn't add it to this pr. Because this bug is so rare, I don't know a good way to add a test for it without changing the code.

[unterkoefler]

I chose the value 0.00001 arbitrarily. Seems like it's big enough fix the bug and small enough to not matter visually. The ruby docs for float link to this page which uses 0.0001 as a delta. Happy to update to that instead if you think that would be better

[gettalong]

FYI In the geom2d gem which heavily uses floating point calculations I have implemented special comparison methods, see https://github.com/gettalong/geom2d/blob/master/lib/geom2d/utils.rb#L24-L34. The precision to use is arbitrarily set.

For Prawn I would probably choose something that reflects the output precision used in https://github.com/prawnpdf/pdf-core/blob/master/lib/pdf/core/pdf_object.rb#L16

[unterkoefler]

Thanks @gettalong!

@pointlessone I updated to a value based on the precision and also updated one other place doing a similar comparison to use the same value

[pointlessone]

I still don't quite understand the root cause of the issue.

[unterkoefler]

I still don’t quite understand the root cause of the issue.

When we render a text box with shrink_to_fit enabled, we call wrap twice. The first time is to check if the text will fit in the box. If it doesn’t fit, the font size is reduced and we try again. When that’s done, the text should fit in the box. Then we call wrap again to actually draw the text.

In wrap, the check to see if the text will fit in one line without wrapping to the next is this:
if @accumulated_width + segment_width <= @width

The bug occurs specifically when the LHS @accumulated_width + segment_width (the entire width of the line) is equal to @width (the width of the text box). The first time through when we’re trying to figure out if the font is small enough, the float math works out such that width of the line <= width of the text box. Everything stays on the same line and shrink_to_fit says that it will fit at the current font size.

Now, wrap runs again, this time to do the actual rendering. This time, because float math is non-deterministic, @accumulated_width + segment_width is not <= @width. It’s very very slightly bigger. So now we think that the text won’t fit on one line, so the last word gets wrapped onto the next line. If the text box is too short, the second line doesn’t get rendered.

The assumption being violated here is that subsequent calls to wrap will produce text laid out the same way.

Also, regardless, @accumulated_width + segment_width <= @width is a problem because it’s comparing floats for equality without a delta.