Appsterdam talk “Chips: The Engine Beneath Your Apps”

Today during lunch, I gave a talk at Appsterdam where I spoke about what’s inside your phone. We start with an overview of the semiconductor industry, and take a deeper look at what makes your phone do all the things it can do: the chips inside.  I made the slides available through slideshare, and Erik Romijn did a quick writeup which you can find on his website.

Marco’s Mobile World Congress 2013 trip report

Last week I again attended the largest mobile show of the year, Barcelona’s Mobile World Congress. Each year, it’s the show I look forward to most, since it’s a great time to catch up with old friends and colleagues who fly in from Europe, the US, and Asia. In addition, the mobile industry is interesting because it’s clearly leading the whole consumer electronics world, and even making a big impact on society and culture in general. Finally, I like the show because it’s in Barcelona, which is a great city altogether, albeit pretty cold this year (5-10C or 40-50F). If you’re ever in Barcelona, make sure you see the roof of the prettiest church in the world. My four-year-old asked if he could just lie on the floor and stare at it while we were there last summer.

About the new venue

Instead of being held at the Fira Montjuic, where MWC was from 2007-2012, it was held this year at the Fira Gran Via, about 4km away from its previous location.

The old venue, especially at night, had lots of charm:

Fira Montjuic MWC

While the new venue lost most of the charm, it doubled the floor space.

mwc fira gran via

What I liked about the new venue:

  • Floor space has become cheaper. Most of the stands (CEVA, Tensilica, Imagination Tech, etc.) were about 1.5x bigger than last year’s. I don’t think their marketing budgets grew by 50%, so the price per square meter must have dropped considerably. I asked several people, but no one could tell me. The big companies (Samsung, Sony, etc.) had even much bigger booths than last year, perhaps 2-3x the size. Small booths should also be cheaper than before. For a small stand at a shared country booth, I heard prices of €1200-2000, which isn’t too bad. The drop in pricing means exhibitors can show more for the same money, so there’s more to see.
  • There are many areas where you can sit down for a coffee or simple lunch. Most of these areas are outside (they call them gardens), are quickly accessible, and had a nice feel to them.
  • Once you get the hang of the layout, you can find things pretty quickly. There’s a central axis throughout the venue, with airport-like walkways, and even though the venue is twice the size, it doesn’t take much longer to get from hall to hall.
  • The entrance is bigger, with many more counters, so it’s much quicker to get into the venue. Hardly any waiting at the door.
  • They had more people on staff outside the venue, in the subway stations, directing you toward shuttle buses, metro stations, trains, etc.
  • Everyone gets a free subway and train pass. No need to worry about public transit tickets.

What I didn’t like so much:

  • The location is less convenient to reach from most hotels. It’s easier to get to the venue from the airport, since they had a shuttle bus that was free, ran often (or maybe I was lucky) and was pretty quick. But from your hotel room in the city, it’s most likely a metro ride and then a train. The trains run often and were a bit full, but it was still pretty quick to get to the venue for me, around 30 minutes, and I stayed close to the famous La Rambla in downtown Barcelona.
  • They need more cloakrooms. While I was able to store my stuff, the lines were sometimes long, and some cloakrooms were full. With a bit of shelving, the cloakrooms should be able to hold twice as much.
  • Wifi access was supposedly better than before, and I got a signal a few times, but pretty much everyone I spoke to had trouble connecting. Wifi still doesn’t scale well for these types of events, unfortunately. It’s ironic that at the biggest mobile wireless show of the year, they can’t get wireless right.
  • The booth numbering inside the halls was pretty bad. You’d expect a chessboard-like A-Z/1-100 naming scheme, but at MWC the letter-number combinations were all over the place.
  • Some of the character of the old venue is gone. The fountains, the view of the palace, the cobblestones, and the meeting rooms that are mostly stowed away in the tower are gone.

Major trends at the show
Maybe I missed it, but there weren’t many big announcements that everyone spoke about. Apple hasn’t attended Mobile World Congress for years, although rumor has it there are quite a few Apple employees on the show floor. Samsung had a huge presence, but just like Apple, they’ll announce their new flagship model at a separate event, this year on March 14. Most of the announcements centered around faster, smaller, bigger, and more of this or that, which isn’t really revolutionary. Here’s a quick video overview from the MWC organization that gives a nice impression of the atmosphere at the conference.

Nevertheless, here are some of the trends I saw:

Hooking up the next billion people

One big focus was on hooking up “the next billion people” to the Internet through low-cost smartphones. There are still a few billion people who don’t have access to the Internet or a phone. Firefox OS targets this space with a lightweight, very open OS that has been picked up by several low-cost models. Nokia presented the very lightweight 301 mobile smartphone. Retailing at €65, it targets people in developing countries and gives them access to a browser, e-mail, Facebook, WhatsApp and Twitter. Deloitte found a direct impact from phone and Internet availability on GDP growth, so this is a very good thing. Everyone in the world should have access to the Internet.

Waterproof phones

Waterproofing simply makes a lot of sense to me. Sony did it in 1984 with the first waterproof Walkman cassette player, and now they’re doing it again with their Xperia Z phone. Plaxo’s study found that nearly 20% of us drop our phones into the toilet. Quite a few more folks find their phones in the washing machine, I’m sure. Waterproofing helps there. In addition, I’m looking forward to reading news on my phone in the shower. Watches have pretty much all become waterproof; let’s do the same for phones.

Full HD displays

It’s been only two years since Apple’s iPhone 4 introduced the retina display to the world. At 960×640 resolution, the iPhone 4 has less than 30% of the pixels of the 1920×1080 phones that Asus, Sony, HTC, LG and ZTE announced, so we’ve come a long way in two years. Even the iPhone 5 at 1136×640 only has about 35% of the pixels of Full HD. It’s only been a couple of years since I traded in my 1366×768 pixel TV to a 1920×1080 Full HD one. And since I’m on a standard cable TV package, I can’t even watch Full HD resolution content on my TV. Soon I will be able to on my phone. It’s another example of how the mobile industry is pushing the whole consumer electronics industry forward.

Computer vision and imaging

At least two hardware companies (CEVA and Tensilica) and a bunch of software companies were showing their computer vision processors and techniques for mobile applications. I’m a firm believer in computer vision, and in the next decade it will turn our mobile phones into extremely powerful information devices. Just point the phone’s camera at an object, and the phone will start showing more info. Think Siri for the camera, or Google Analytics for video. Contact me if you’d like to talk about this; I’m quite involved with this technology and would be very interested to discuss and explore further.

Operator commoditization

For many years, mobile operators made their money on voice, text messages, financing your phone, and data services. Since voice and text messaging are quickly going away as a source of income thanks to WhatsApp, Skype, Google+, e-mail, etc., the operators are afraid they’re turning into commoditized dumb-data-pipe providers. As a result, they’re trying to control the experience of the phone much more, and they’re all trying to find new revenue sources. They’re scrambling. One example is Vodafone, Telefónica and Telecom Italia’s urge for regulators to allow consolidation in the industry. “Mobile operators in most markets are suffering from too much competition.” I don’t think competition is the problem; commoditization is.

Wrapping up

Another novelty is that getting a room through Airbnb instead of at a hotel is gaining traction. Using Airbnb, it was easy to find an affordable apartment (€60/night) in a pretty central location (close to La Rambla). A friend of mine even paid €25/night and was happy with the room. Hotel rooms on average ran €250-300/night. Sure, it’s a bit of a gamble, but I’ve found Airbnb to be a good alternative to hotels.

The MWC Android app was pretty horrible. In order to get a map of the show floor with its exhibitors, you needed to be online, which means mobile phone costs.

That wraps up my show report. It was great to catch up with you all, and we’ll see you again next year. I’ll be there for sure.

Mobile World Congress 2013

It’s that time of the year again, only a couple of weeks to go until the Mobile World Congress in Barcelona kicks off. I’ll be meeting many of my friends and old colleagues there, it will be great to catch up with everyone.

Some of the things I’m looking forward to:

  • Last year Nokia introduced their 41Mpixel camera phone. What’s new in camera land this year?
  • Huawei about doubled their presence each year, what did they do this time?
  • The Dutch Mobile Networking Event – the Dutch party to be at, run by Caroline Spek
  • What’s new in accessories? Will we see new wrist accessories? What’s next, a bluetooth ring?
  • How many new phones will be waterproof like Sony’s?
  • What’s up with the latest displays? Can we bend them? Will we go higher than full HD for mobile? Anything new that lowers the power consumption or increases the quality?
  • Is anyone able to challenge ARM? They’re pretty much a monopolist in mobile.
  • Will Imagination Tech position their recently acquired MIPS products for mobile?
  • How is mobile app development evolving?

Looking forward to see you there. Please don’t hesitate to contact me in case you’d like to meet.



They’ve done it again: video files again lost half their weight.

Some of you probably remember video discs (which used MPEG-1/H.261 for video compression), then there were DVDs (MPEG-2/H.262), then Blu-rays (MPEG-4 AVC/H.264). Between DVD and Blu-ray there was another video coding standard, the original MPEG-4, which didn’t make it onto optical discs, but was used quite a bit also. Each time, a bag of new algorithmic tricks meant that the video could be compressed twice as much.

Loosing weight is hard. In video compression, it’s quite the same. Just when you thought video can’t be compressed any further, there’s a new standard that’s done it again: HEVC compresses video at twice the rate of the preceeding standard. This means double the hours of video that can be stored on your disc, drive, stick or card. Double the number of channels broadcast over the air. Or half the download time. Or half the data usage on your phone plan. We’re clearly in anorexic territory here, and for once, I like it.


Super cameras out of a spray can

Researchers from the Technische Universität München have developed a new generation of image sensors that are 3x more sensitive to light than the conventional CMOS versions, with the added bonus of being simple and cheap to produce. These sensors can be manufactured without the expensive post-processing step typically required for CMOS sensors, which involves for example applying micro-lenses to increase the amount of captured light. Instead, every part of every single pixel, including the electronics, is sprayed with a liquid polymer solution, giving a surface that is 100 percent light-sensitive. Changing the chemical compound turns the sensor into an infrared sensor for night vision.

In short, better and cheaper cameras that also work at night. Me want.

Read the full story

Retailers watching customers with eye socket cameras

Interesting article at Fast Company

As if store mannequins weren’t creepy enough already, shops like Benetton are peppering their storefronts and shop floors with dummies that have a camera embedded in a single eye socket, that will watch while you shop.

They’re at eye level, and customers tend to pay attention to them more.

I’m used to exactly the opposite: showing off the security cameras to deter shoplifting. The eye socket cameras are much more tricky, and are a clear step toward a future where cameras are embedded into many objects around us. Be prepared to be watched.

Continue reading

Very very small pixels

Researchers at Stanford University are developing a multi-aperture image sensor which groups arrays of 16×16 pixels, then puts a tiny lens on each group. Their 3Mpixel image sensor in this way includes a total of 12,616 lenses, compared to a shabby single lens commonly found in cameras. The benefits are plentiful. The simpler electronic design means the pixels can be 0.7um, much smaller than Kodak’s 1.4um pixels that I posted about earlier. Camera modules incorporating this technique can be made even smaller, cheaper, more robust, and, most importantly, grab better pictures. Instead of taking a single snapshot, the camera actually takes 12,616 pictures, which can be combined with digital image processing techniques to capture 3D image data, to accurately control depth of field, focus, etc. With enough image processing power available in the camera, this opens up a whole world of new possibilities.

A high level overview of the work can be found here and their technical ISSCC paper can be found here.

Very small pixels

A few weeks ago, Kodak announced their new 5Mpixel image sensor at the Mobile World Congress. The sensor has a 1.4 micron pixel size (1.4 by 1.4 micron) which means the sensor can fit in a 4x4mm camera module. That is about the size of a regular black ant. I am sure a bigger ant could carry such a camera. The Kodak sensor has some novelties. There is a new color filter pattern, which includes a “white” photocell receptor instead of just measuring the amount of red, green and blue. That will require quite some changes to the image processing algorithms. Another novelty is that the sensor measures darkness instead of light. Apparently that can be more accurately implemented in silicon. Like most new sensor introductions, Kodak promises higher quality images than anyone else.

Micron just announced that it spun its image sensor business out into a new company called Aptina. The business will be run by Micron’s Bob Gove, who was previously at VLIW processor company Equator. Micron says they have already sampled an even smaller 1.2 micron pixel, which in the same 4x4mm tiny camera module would yield a 7Mpixel sensor.

Tested pixels

I just ordered a new camera for personal use. It’ll be my first SLR. Most cameras I’ve held, either in the office or at home, I have pointed to this chart to test the camera:

You can get the original from Stephen Westin in pdf here. Simply printing it on any decent laser printer does a pretty good job. If you have an A3 printer, even better. It’s interesting to see that most of today’s camera phones don’t even do a low-pass filter before subsampling on the viewfinder, causing bad aliasing. There’s still lots of room for improvement!

Display pixels versus capture pixels

One of the artificial questions I’ve been pondering a bit is what we will see more of: pixels that capture light (cameras) or pixels that make up displays? For several years my prediction was that soon we’d see more cameras than displays in the world. The reasoning was that displays are relatively large and made to be seen by a human. Cameras are tiny though and have many uses. Cameras don’t need humans to look at the images captured, they can simply be stored, or analyzed by an algorithm running on a piece of silicon. Since there’s only a little over 6 billion of us to view the screens, soon we’d have more cameras than displays.

Some cell-phones include two cameras, one for video conferencing and one for taking snapshots. These only have one display, which confirmed that I was right. Soon we’ll have more cameras than displays.

Still, the other day I saw a very small digital photo frame which cost only 15 euro and was meant to be worn on a key chain. It could hold 30 photos or so and contained a tiny battery. This caused me to think that the cross over point of having more cameras world-wide than displays is quite far away. We’ll soon have displays on our credit cards, on the outside of our laptops and perhaps even on our clothes.

Do you think we will ever have more cameras than displays?