Integers in PHP, running with scissors, and portability

Until recently I thought that currently popular scripting languages, which mostly evolved over last 10 years or something, must allow for easier portability across different platforms compared to ye good olde C/C++.

After all, their development started a few decades after C, so its notorious caveats are all well-known and should be easy to avoid when designing a new language, right?

However, PHP just brought me a new definition of “portable” – and that was when working with… integers.

PHP is not able to handle unsigned integers, and converts values over 2^31 to signed. So if your IDs go slightly over 2 billion, and PHP decides to treat them as integers, you’re in trouble.

Oh wait, no – that’s on 32-bit platforms only! PHP int size is platform-dependent, and it seems to be 8 bytes on our 64-bit boxes. Yes, the very same ones where C/C++ int is 4 bytes, you know.

That was the easy part. It was mostly documented.

Now, there’s a function called unpack() which essentially allows to convert different types of data from binary strings to PHP variables. What if you try to unpack unsigned 32-bit big endian integer (format code “N”)? Let’s check the doc:

If you specify a number beyond the bounds of the integer type, it will be interpreted as a float instead.

Having read the doc I personally blatantly relied upon it and expected that large unsigned 32bit numbers would be converted to float, or string, or something, but handled properly. However, a couple or so weeks ago the following notice suddenly appeared:

“Note that PHP internally stores integral values as signed. If you unpack a large unsigned long and it is of the same size as PHP internally stored values the result will be a negative number even though unsigned unpacking was specified.”

How sweet. No, it just could not behave like documented and convert 32-bit unsigned value to float on x32 or keep it integer on x64 – you now suddenly have to care about value size yourself. Ah, and by the way, there’s no official way to know what’s int size.

To make things even better, 5.2.1 introduced a nice bug in unpack(), which f..ed unpacking less-than-16-bit values on x64. (I assume you understand that “f..ed” means “fixed”). It took some time and several tries to convince PHP team that x64 has enough bits to hold 16-bit unpacked value, but thankfully its now acknowledged and assigned.

To summarize, if you need to unpack an unsigned 32bit int from binary stream, you have to:

• convert it to float or string manually,
• do that depending on int size on current platform,
• which can not be done using anything documented,
• and specifically avoid PHP 5.2.1 on x64.

Most people could probably learn all that, and then use sprintf(“%u”,$id), work with string IDs everywhere, avoid 5.2.1 and be happy.

Unfortunately, my final goal was to have support for 64-bit document IDs…

Let’s do a small time travel. Integer types in C/C++ have always been a pain, but back in 1999 ISO commitee ratified ISO/IEC 9899:1999 standard, also known as ISO C99, which guarantees that “long long int” integer type must be at least 64 bits in size. By now, most compilers support that part perfectly.

However, designers of PHP 5 (released in 2004) type system were either not aware of this change, or decided to not rely on the standard which has been out for “only” 5 years by then, or just thought that 31 (no typo) bits and 640K should be enough for everybody.

Long story short, it’s 2007 now but there’s no native 64-bit integer type in PHP. Let me remind that built-in “int” might be 64-bit, but then again it might be not, and there’s no official way to tell.

This time, there’s a number of routes one could take – either use ints (and pray that the app is never run on x32, and that “platform dependent” size does not change to 4 next version); or use GMP or bcmath extensions if they are available.

Fine, so 99.999% of the world would hit that, compile in bcmath, and be happy again.

Unfortunately, I needed to develop a library which could be deployed in any environment – and still work, and produce reasonable results. The worst case is x32, and neither GMP nor bcmath available.

And this is how the following code was born.

For reference, this is what would the equivalent C/C++ snippet look like:

Portability in year 2007.