The CharlieCard Screen of Death

Seems the software behind CharlieCard readers was built in Microsoft Visual C++. And guess what? It's not immune from crashing. Zeroday posts the photographic proof from the Central Square station.

Neighborhoods: 

    Topics: 

    Free tagging: 

    Comments

    That's a text string

    By on

    That's a text string formatting error. As I don't know which version of Visual C++ they are using, I can't tell any more from the error, but it should have been caught in testing. It shows sloppy programming (they let Visual C++ produce a crash, rather than check for the possibility of errors), a lack of testing, and just general bad design - running the CharlieCard system on Windows? Come on.

    Journocidal Organization

    If anybody had found and reported that sort of bug, much less expected it to be corrected, they would have been vilified, and the T might even have sicced the legal department on them!

    Buffer overflow, perhaps?

    The C sprintf() function is infamous for not checking the length of the buffer it is writing to (because it can't). This can easily cause a crash or memory corruption. A safe application probably should be using snprintf() instead. Or Java, or C#.

    They distributed the program

    By on

    They distributed the program with the debug libraries, too. Guess they aren't worried about speed.

    I suppose one should be thankful - they'd never know where to look for the bug if not for that ...

    Debug libraries are common

    I always included them in projects where we owned all the hardware. In theory, by the mid-1990s, there was only a slight performance hit, and the gains from getting full debug data significantly outweighed the cost of losing the more esoteric compiler optimizations - especially since disks were still too slow to do useful transaction logging.

    On today's even faster systems, I doubt it matters much at all. I can't imagine that turnstiles are CPU-bound.

    Hey, wow! So they got something right!

    hardware error, not software?

    By on

    ...because the system doesn't do anything different depending on the card. It has a standard set of messages it displays.
    Sounds to me more like bad memory on the gate computer...

    Oh...wait...it does display the remaining card/ticket balance. Hmmm.

    Charlie gate actions

    The gate certainly does different things depending on the contents of the card/ticket. Most notably, it either opens or refuses to open. In the latter case, it might display a message that there is not enough value, or that the card/ticket is expired, or that it is unreadable.

    Also, if the card or ticket has a monthly pass, the gate displays the expiration date when it opens.

    Gates also have to properly deal with transfers from recent bus trips.

    no shit, sherlock

    By on

    The gate certainly does different things depending on the contents of the card/ticket. Most notably, it either opens or refuses to open. In the latter case, it might display a message that there is not enough value, or that the card/ticket is expired, or that it is unreadable.

    Siiiiigh. Yes, but it only does so from a finite set. There's nothing UNIQUE printed, except the value on the ticket or card. EVERYONE gets one of the options you mentioned.

    Thus, the error described is somewhat unlikely to be a programming error, or every machine would be doing it, on a regular basis. Like I said, hardware failure seems a more reasonable explanation. Or, maybe a card/ticket got scrambled in some way, and the error (as mentioned by others) shows improper error handling and/or data validation.

    Law of large numbers = bugs turn up

    That's absolutely not true; there are plenty of software errors that can turn up, years later, on "frequent but finite" type systems.

    It could be an unexpected input, as you mention. We also don't know what the back end looks like; are there configuration files that get occasional updates from the server? Are there metrics or transaction logs being written out, where some value might overflow?

    Maybe this bug happens the 32768th time the turnstile has to reopen the TCP socket to the central server. Or it could be a race condition; you get LOTS of those on what you'd otherwise expect to be a completely deterministic system. Maybe it happens when more than 16 people simultaneously go through the other turnstiles, and two of them have unreadable cards, and one other hits $0 exactly, and the back-end bus is hanging on a spin lock and times out.. and so on.

    My favorite bug was a completely repeatable server crash that happened when an AOL user, using a DOS (not Windows!) client (2% of the user base), tried to read an e-mail from the Internet (50% of e-mail), which was longer than 8K (<2% of e-mail), and in which the dotted line separating the headers from the body fell across the 8K boundary (<0.??% of e-mail).

    It happened once every few weeks. And then once a day. And then a few times a day. And then a lot. And then I fixed it, but not before a few sheepish IMs saying "Uh, hi there, excuse me... Sorry about that hourglass, but what button did you just press?"

    No, it's a software error

    By on

    The program is formatting some kind of data - either to write it to the screen, or to a file either on the system itself or on the server.

    Correction

    By on

    cprintf() is for writing to the console - not to a file.

    Again, a really dumb error.

    Re: The CharlieCard Screen of Death

    By on

    Yes the Charlie Card / Ticket machines run on Windows. Soon after the system wide roll out of the automated fare system in early 2007, I noticed a simultaneous re - boot of all the ticket machines at Porter Sq. They all started displaying the standard XP boot time splash screen.

    What's even more interesting is that Windows appears to be imbedded in the fare gates also. A few months after my Porter Sq. observation, one of the gates at Wellington would not swallow my weekly pass. When I glanced at its LCD display, it was showing the XP system management console! A week later on yet another out of order fare gate at Wellington, I noticed a Windows file folder tree displayed!

    I wonder if the same is true of the Green Line / Bus fare boxes - did the T deploy Windows everywhere?

    ~Hakim~