technology · Grey Nicholson

Self-driving cars: software crashes

2018-02-25T23:28:00+00:00

No car is self-driving.

A “self-driving” car is piloted by software, which is ultimately written by a person. You don't know who that person was; only that they were employed by a particular company.

They were probably sitting in an office somewhere in California when they wrote the code driving your car. Maybe it was 17:30 on a Friday and, despite caring sincerely about the work they were doing, they happened to be distracted by the prospect of going home. Maybe not. You don't know.

Do you trust that person with your life?

Well, the company hired them, so they can't be completely useless. You trust the company's recruitment procedures. …What are the company's recruitment procedures?

Anyway, presumably there are processes in place to review the code, and stop mistakes from making it into the final software. Presumably. You trust that there are, and that they work, and never fail.

Now imagine the company has made it illegal for you to see how the software works. Are you sure you trust this company with your life?

There should be a law saying that if a vehicle can be piloted by software, and it's capable of containing or hurting a human, then all installed software must be open source, and you must be able to prove that the source code corresponds to the software running in the car.

It has to be legally possible for the vehicle's owner (or prospective owner) to discover how their car might behave in a life-or-death situation, so they can decide whether they want to be responsible for the car's actions.

Responsibility

Logically, the manufacturer who wrote the software would be responsible, but they have no incentive to take responsibility for their cars' imperfections. Doesn't make money. Why admit your own flaws while your competitors keep schtum, look better, and rake it in? Any goodwill from better transparency will evaporate as soon as someone dies in an accident.

It's much safer to claim that the human pilot should have taken control at the critical moment. Capitalist governments won't argue with rich, profitably-taxable businesses.

Car makers will only be transparent about how their cars behave if they're obliged to by law.

Open source

Merely having access to the software's source code isn't enough. It must be legal to reuse the source code, for several reasons.

Morally, if Non-Specific Engines Ltd writes an algorithm that's better at saving lives than any other algorithm, shouldn't Acme Motors be obliged to used the safer algorithm in their cars, rather than forbidden?

Practically, you need software experts to audit the code. You want the code checked by an independent expert in the field of vehicle automation — not a business partner of the manufacturer — and that person will be a software developer.

If they use a similar concept in their own work later, Mom's Friendly Car Company could threaten to sue them, claiming they copied the code illegally. Software developers are rarely as rich as car companies; even the threat of a lawsuit would mean that in practice the code would go unchecked.

And again, morally, you can save lives here, by letting the developer reuse the good code.

Reproducible builds

Lastly, it needs to be possible to prove that the audited code is actually the code running in the car. You want an independent auditor to build the software for themself, in a development environment they trust, and get the exact same output as what's in the car. It must be possible to build the software reproducibly.

Otherwise checking the code is pointless — you still have to trust the car manufacturer, and you can't be sure the software's behaviour doesn't deviate in subtle ways in very specific situations. Maybe you don't care about any subtle differences, but maybe you do. The driver should at least be honest with you, and you can decide for yourself.

None of this will make sure a self-driving car is perfectly safe. All software has bugs. But at least you'll know the driver was acting in good faith.

Trade secrets and competitive advantage are not worth dying for.

…Or you could just trust the big friendly company… right?

Into the Fire

2009-03-06T15:07:00+00:00

An entry at Illuminated Mind has struck a chord with me. It's titled “If It’s a Good Idea… Don’t Do It”, although “If It's Just a Good Idea…” captures the message more clearly.

One of the reasons I petered out of university was that I was doing it only because it was a good idea, and not because I was still passionate about understanding the physics. Even the dead-centre-of-my-field-of-interest module of Cosmology was failing to inspire me.

It wasn't at all arduous: I still came upon flashes of epiphany every so often—one such I recall involved the insight that the constituent particles in atoms' nuclei are also arranged in shells, like the orbiting electrons, and that shell boundaries could explain some discontinuities in each element's differing physical properties—but they were too few and too far between to hold my interest.

I also realised that I wasn't especially good at maths, and the solid intuitive understanding of maths needed to grok the physics I was learning was just slightly beyond me. Perhaps I stopped caring about maths too.

I realised that I only cared about qualitative trends, and not about the process of calculating results from formulae, or being able to properly derive a formula from memory and a set of more fundamental equations. Once I knew that formula C necessarily followed from formulae A & B, I was happy. I still like unscaled graphs as qualitative illustrations.

The main thing I cited at the time was Six Feet Under. More4 were showing repeats of the later series late each weeknight. I decided that the characters' philosophical discussions (perhaps combined with the late hour of the broadcast) were expanding my mind more than rigorous study of science.

I never particularly wanted a degree—I just wanted to learn about the universe. And I found myself gaining more insight into the universe by following the fictional escapades of a family of intelligent undertakers than by computing physical quantities.

So I was doing what was a good idea, rather than what inspired me. I'd made a similar decision before, when I chose to carry on Religious Studies to full A-level in college, instead of continuing to study Chemistry, despite doing slightly better at chemistry in practically every respect.

The difference was that in that case there'd been a fire for me to jump out of the frying pan and into. Watching Six Feet Under is not a full-time occupation.

So I bummed about a bit, not even looking for a job for the next few months as I had some spare student loan and overdraft to play with.

I eventually happened upon a desk job that suited me down to the ground: opening letters, filing, sorting and being a general administrative office bitch, for the customer relations department of a train operating company.

The work itself was easy and almost entirely stress-free, as I always had someone to turn to when in doubt. I continued to devote much of my mindshare to the day-to-day happenings in software and technology news (an interest I can trace back to the buggy implementation of CSS in Microsoft Internet Explorer 6).

I did, of course, think about my job as and when required. As is my wont, I came up with a couple of suggestions to do things better in the office, but anything beyond keeping tidy, labelled piles of stuff was stymied by being stuck with a particular set of tools.

This was OK. After all, I didn't really care that much. Yes, it'd be nice if Mrs Smith from Glasgow could receive a response to her complaint about delayed trains a day or two sooner, but.

I was rudely awakened from my employmentary coasting when my office was relocated to Newcastle, a 60-minute (free) train-ride each morning and evening, on top of the twenty-minute walk. This gave me plenty of time to read some books for a change, and I sped through The Moon Is A Harsh Mistress by Robert A. Heinlein and the original Foundation trilogy by Isaac Asimov (in full this time) over the course of a couple of weeks.

But co-workers and my genial manager were leaving because of the move, and the immediate vicinity of Newcastle Central railway station is no match for the lunchtime scenery afforded by York's River Ouse and Museum Gardens, especially in summer. Though I couldn't see myself leaving, I couldn't see myself staying there either. And so I jumped: I handed in my notice (metaphorically—I actually just told the outgoing and incoming managers when I intended to leave).

Technology news was replaced by job-searching. I recall one Tuesday in Newcastle spent, by all three members of the administrative team, primarily looking for other employment.

Astoundingly, it worked, and two weeks later I was staying in a hostel in Bristol about to be trained to be a charity street fundraiser.

Thirty-six hours after, I found myself standing at the side of the main street in Leeds, nervously sipping at a cup of tea, trying to dilute the adrenaline.

That day went unexpectedly well, propelled largely by my frequent self-reminders that I was talking to random people on the street and trying to convince them to donate to a charity, for a living.

Three months elapsed.

My old teammate had often told me that the day's outcome could be influenced by my expectation of what would happen. I expected to come back to work and do solidly well. I expected that I knew what I was doing. My old team-leader had said he firmly believed that if you wanted something enough, you'd find a way to attain it. I believed that I wanted to be a good fundraiser. The team's coach insisted I had it in me. I deferred to his superior understanding.

It was starting to become apparent that I wasn't meeting targets. Not just the actual targets, but the minimum ones too. I wasn't earning my place on the team.

This came into focus one Wednesday evening in York when, after applying every technique I could muster—level thinking, playing the long statistical game, appropriate body language, concision, knowledge, friendliness…—I found myself walking home without having signed a single person up.

I realised I didn't actually know what I was doing. I was doing a good impression, much of the time, of someone who knew how to go about street fundraising, but in reality I was winging it. And not well enough. An awful day should yield no fewer than two sign-ups; the remainder of the team had signed people up.

I arrived largely undeterred the following morning in Newcastle, after a journey that was equally oddly familiar and strangely different. It was drizzling—not supposed to affect a fundraiser's performance, but not especially conducive to it. As it was November, there was an almighty racket accompanying a department shop's Christmas display, rendering a decent swathe of the street useless for talking to people. I was deterred.

Despite this, I signed two new donors before lunch, a decent tally for the morning session. Briefly, I convinced myself that I'd remembered how to fundraise. I spent most of the afternoon mulling over the decision to quit, safe in the knowledge that soon enough a decision would most likely be made on my behalf anyway. I even signed another person up (though I'm told he's a serial charity-joiner).

The following morning's news headlines centred around the deepening recession: specifically around new figures for job losses, and the recession's stifling of charitable donations. I have a wry sense of humour.

Jonathan's blog post reminded me strongly of when I left both university and fundraising. Both times, the initial spark of enthusiasm for what on paper is a pretty awesome idea had dwindled, leaving me merely going through the motions.

University lectures were not a chore, so the twenty-or-so hours of every week of term that they occupied weren't sorely missed. Even spending a day in labs each week was OK. Attaining a proper quantitative understanding of physics, though, required plenty of off-timetable study—time I was loath to put in since in truth I cared little for the specifics. It showed: each week's problem questions seemed more and more daunting, which only increased the resolve needed to actually study.

Fundraising had me leaving home each day at around 07:30 and getting back at about 22:30—albeit typically due to healthy after-work socialising. (And I should point out that everything that went with the job was wonderful: my teammates, and our adventures together—even running for a train back from Harrogate with one minute to spare felt like an adventure—were each uniquely brilliant).

I now know that I'm not capable of devoting my time to something that I'm not enthralled by. It doesn't have to be actually important—I just have to care enough to want to take control.

I suppose part of it is the knowledge that someone else could be doing a better job instead of me. My demotivation in physics and fundraising alike roughly coincided with the dawning that I would be neither a brilliant cosmologist nor a legendary fundraiser respectively.

I have this need to correct things and improve upon what's there, and if I can't make something better I tend to leave it to someone else.

So I agree with Jonathan that something just being a good idea doesn't make it a useful application of my time.

The BBC Misunderstands DRM

2008-01-30T16:08:00+00:00

With the furore over the BBC's unfair use of DRM in its iPlayer, one would expect them to know what DRM was. Apparently, they don't.

(At least, their technology editor doesn't.) BBC News has an article about “a new method of DRM” being used by indigenous Australians to make it easier to respect their societal customs. The thing is, the system as the article describes it is not DRM.

DRM is where a file (a document, some software, or a piece of audio or video) is designed to be unreadable without a key (a cipher—like a password). Crucially, the key must remain unknown to (or unusable by) the file's intended audience.

The DRM acts as a gatekeeper, and only allows access to the file (lowers the drawbridge) if the audience can demonstrate that they are entitled to access the file. The audience may only access the file if the key-holder allows it.

Making a copy of a DRM-encumbered file and giving it to someone else typically results in their not being able to use it, because the DRM recognises them as someone else. (If it doesn't recognise them at all, they still won't be able to use the file.)

That's the intention, anyway. It's not actually possible to implement DRM so that it works—it can only ever make viewing the content a bit more awkward, for determined copyright-infringers and ordinary consumers alike.

The key point here is that the DRM—and thus the key-holder—controls the audience's ability to access the file.

In the situation described by the BBC article, the audience chooses to avoid certain content, based on a set of cultural rules. The “DRM” system allows the audience to filter the content, by taking cultural information about the audience and applying those rules.

There is no key (that the article mentions, anyway); no-one is prevented from accessing any content—it's just made easier to avoid content that's inappropriate for that particular person.

This system sounds rather more like a simple set of filters that work on metadata associated with each bit of content. That's a lot more like a porn-&-swearing blocker than DRM. It's very similar to Flickr's search engine.

Indeed, it's very similar to any search engine, except the criteria are “appropriate for this age, that gender and this community” instead of the more typical “must contain the words foo, bar and baz”.

So this is, as the article describes, an example of applying search logic and technology (that itself originates from European-influenced cultures) to solve a cultural problem that many from European-influenced cultures would find surprising.

It has nothing, however, to do with DRM (which is an example of unfeasibly clinging to a business model that's been made obsolete and unworkable by technological improvements), as the article erroneously suggests.

The New Illiteracy of the Internet

2008-01-02T00:54:00+00:00

Kottke writes about a 56-year-old prediction of a positive effect from television on literacy, and notes that predictions for “television” closely resemble the modern web.

A lot of modern communication technology is textual which, a few decades ago, when television and home video were at their height, would have seemed odd. But it turns out that text is more efficient than audio and video. I think this is because basic literacy levels have improved: people are generally expected to be able to read and write text, which has made text-based technology convenient, and has also improved the rate of basic literacy.

This means that lots of people can use text to communicate; it most certainly doesn't mean that those people are using good-quality written language. Ppl r in ur intArnet, revelin in intenshunly wierd gramr. It's reminiscent of an intentional misspelling fad in nineteenth-century America, from which the word/phrase “OK” is suggested to have arisen.

Where, when I was a lad, kids would only use text when writing for their teachers, who would then correct and frown upon misspellings and poor grammar, now children talk amongst themselves using text—internet instant messaging, mobile phone–style text messages and emo blogs being the primary culprits. (Be nice to LiveJournal—it's felling lonley rite now.)

So people get used to using unconventional or incorrect (depending on your viewpoint) spelling and grammar, with the understanding that the receiving party will nonetheless be able to understand the message. (This is compounded by a general reluctance to correct or be corrected.)

Where this compresses communication, for example by abbrev.—making it quicker and generally more efficient—this is not a bad thing. There are a set of essentially universally–recognised abbrev's, e.g. “e.g.”, “&”, & “etc.” etc.; numeral figures and mathematical symbols can also be considered examples. A problem only arises when meaning is misinterpreted.

Some help for English-learners and by way of an example: more often than one might expect, “then” actually means “than”. “More often then one might expect” doesn't actually make any sense and “then” sounds similar to “than”, so most experienced English-speakers can understand the message. “There”, “their” & “they're”, and “to”, “too” & “two” are two other classic examples of words being conflated.

My point is that whereas before the advent of recent technology a smaller number of people had a greater quality of literacy, now a greater number of people has a lesser quality of literacy. It's as if the ubiquity and the quality of literacy sum to a constant.