# Friday, 15 August 2014

Android NFC service and "thin client": one problem, and one hack

Lately (in the last year or so), Android work intensified at my company. So, I finally took the time to study it in depth and I discovered how MUCH Android differs from what I was expecting. It really starts to make sense when you dig under the cover. And you start to discover how much better your apps behave when you are using the SDK the way it should be used (and you also start to pick up defects in other apps and say "Ha! You did that! Gotcha!).
But this is the topic for another post... :)

Today I want to concentrate on an issue I was experiencing using the NFC framework in Android to read our contactless cards.
Using the NFC framework as a general purpose card reader is a bit on the "stretchy" side: the framework, after all, is mainly there to read Ndef tags, which have a precise structure. Fortunately, Android allows you do go deeper, and interact directly with a card using a "transceive" method.

In this way, you can send commands and data to the card (in the form of a byte[]) and receive a response from the card (again, in the form of a byte[]).
So far, so good: this means that we can read our Mifare Desfire cards, using the Desfire authentication and our keys.
I implemented the commands to authenticate a card, select an application (i.e. a protected directory inside the card memory) and read the data.

All is working well, but.. you have to store your key on the phone, and the storage on your phone is not secure.
In theory, every application has a local storage that cannot be read by other applications. In practice, you just have to have root access to your phone (which is mighty easy with Android handsets) and you are done.

This is not a particular problem for some scenarios (e.g. if you provide an app that uses the user differentiated key, so that the user can read his own card), but it is a problem when you need to read multiple cards, and therefore to use the master key.

Suppose you are a third-party company. You are my friend, and you want to provide a discount for my subscribers (people that have my smart-card).
How can you check that the card is real, and that the card is not expired? Easy, you authenticate with the card and read its content: the expiration date is written right there.
But I do not trust you enough to let you have my read keys!

Maybe you even want to top up my card with "reward points": if my users buy something from you, they will have discount on my services. Super cool!
But I will not let you have my write keys.. that's out of question!

Sure, you can read just the UID, and use that to look up user info on my web-service. And use the same service to POST reward points. But my network is sparsely connected, and it might take long before a card is used on one of my terminals and I can update them.
And we have seen that a UID can be faked..

The answer is "thin-client". You use your NFC phone as an antenna, nothing more. What you read from the card is sent as a (hex-encoded) string to a web service. The web service contains the logic and data to interpret the request and prepare the right response. The response is sent back to the phone, and then transmitted to the card.

You can authenticate with the card, but your keys are safely stored away on your server and they never transit on the the phone!
The phone does not even see the personalized key, so the user is safe against cloning.
I build a prototype, and it worked great on our WiFi network.
They I tried to use it on a cellular network and it failed (almost) regularly. Why?

My suspect was that after a (very short) while the card was reset.
The answer I was getting back from the card was something like "operation not supported in this state". It was like somehow the card forgot that we were in the middle of an authentication challenge-response before the protocol was over.
I decided to investigate, to see if my suspicion was confirmed.
Fortunately, Android is OSS and source code is available! So I dug into the Android source code, looking for clues in the NFC implementation.

Android implements NFC using a mix of libraries and processes; most of the NFC stack is native, and managed by the OS. Then, there is a Service (provided by the OS) that handle communication with the native NFC stack. And some client-side classes you can use inside your application, which will communicate with the Service, hiding it from you.
I started to dig into the source by following a "tranceive" call.

On the application side, you receive an Intent when a card is presented to the reader. Inside the intent payload there is a class derived from BasicTagTechnology; in our case, we use a ISO-A compatible card, so we get a IsoDep object.

The most important method of this class is, as I mentioned, tranceive:


The method inside is just a thin wrapper for remote invocation to a service, which is the NfcService or NfcApplication (the name has changed between Android releases:

Tag.getTagService().transceive(mTag.getServiceHandle(), data, raw)

class Tag ...

    public INfcTag getTagService() {
        return mTagService;
INfcTag is an aidl interface, which is used to forward data and commands to NfcService.
We can follow the transceive implementation inside NfcService:

public TransceiveResult transceive(int nativeHandle, byte[] data, boolean raw)   
 tag = (TagEndpoint) findObject(nativeHandle);
 response = tag.transceive(data, raw, targetLost);
 Object findObject(int key) {
        synchronized (this) {
            Object device = mObjectMap.get(key);
            if (device == null) {
                Log.w(TAG, "Handle not found");
            return device;

So, there is another "Tag" class inside the service; all known (in range) tags are held by the NfcService class in a map.
This "Tag" is named NativeNfcTag:
public class NativeNfcTag implements TagEndpoint
   private native byte[] doTransceive(byte[] data);
   public synchronized byte[] transceive(byte[] data) {
      if (mWatchdog != null) {
      return doTransceive(data);

The implementation of doTransceive is native, and it varies from a card tech to another.
We have found the end of the flow. Have we also found any clue about the card reset?

The answer is there, inside NativeNfcTag. You should have notice the "mWatchdog.reset()" statemente inside doConnect. What is mWatchdog?

private PresenceCheckWatchdog mWatchdog;
    class PresenceCheckWatchdog extends Thread {

        private int watchdogTimeout = 125;


        public synchronized void run() {
            if (DBG) Log.d(TAG, "Starting background presence check");
            while (isPresent && !isStopped) {
                try {
                    if (!isPaused) {
                        doCheck = true;
                    if (doCheck) {
                        isPresent = doPresenceCheck();
                    } else {
                        // 1) We are paused, waiting for unpause
                        // 2) We just unpaused, do pres check in next iteration
                        //       (after watchdogTimeout ms sleep)
                        // 3) We just set the timeout, wait for this timeout
                        //       to expire once first.
                        // 4) We just stopped, exit loop anyway
                } catch (InterruptedException e) {
                    // Activity detected, loop
            // Restart the polling loop

            Log.d(TAG, "Tag lost, restarting polling loop");
            if (DBG) Log.d(TAG, "Stopping background presence check");

The "watchdog" is a thread that at short intervals (125ms) checks if the card is still in range, using the "doPresenceCheck()" function. Which is native, and card-dependent.

The function could be therefore an innocuous instruction (a no-op), or a new select that will reset the card to its not-authenticated state.
Guess which one is for Desfire cards?

So, if the watchdog is not reset periodically by transmitting something to the card, a presence check will be triggered and the card will be selected again, resetting the authentication process. While you are still waiting for the cellular network to answer (125ms is a short time on 3G).

I started to think on ways to work around it, from suspending the thread (inside another process - the service - in Android? Root necessary), to set the timeout (by invoking a method on NativeNfcTag using reflection... again, another process was out of my reach) to substitute the code for "doPresenceCheck()" (which you can do with things like Xposed, but.. that will require Root access too).

You just cannot access anything inside another process in Android, if you don't have root access. Which is usually a very good thing indeed, but it getting in our way in this case.
But what about our process? Sure, we can do almost anything inside it... but how can it do any good?

Well, there is a function inside NativeNfcCard which we can use. This function is "exposed" from "Tag" (the non-public class used at "client" side, see above), but not by BasicTagTechnology.
So we cannot call it directly (like transceive), but from the Tag class onwards it follows the same flow as transceive. This function is "connect":

class Tag {
   public int connect(int nativeHandle, int technology)
   public synchronized int connectWithStatus(int technology)

If we examine the source code of "doConnect" on the other side (its implementation inside NativeNfcCard) we can see that this function will reset the watchdog too (like transceive). Moreover, we can turn "connect" into a no-op:
private native boolean doConnect(int handle);
    public synchronized boolean connect(int technology) {
        if (mWatchdog != null) {
        boolean isSuccess = false;
        for (int i = 0; i < mTechList.length; i++) {
            if (mTechList[i] == technology) {
                // Get the handle and connect, if not already connected
                if (mConnectedTechnology != i) {
                } else {
                    isSuccess = true; // Already connect to this tech
If the technology we specify is the same one we are already using, or if it is a non-existing technology, the function will do nothing.

We can just grab the Tag class inside our code, call connect on our side (using reflection, as it is not exposed by the API), and wait for it to forward the command to the service, resetting the watchdog. Do this regularly, and we can "buy" as much time as we want to complete our authentication protocol!

This is obviously a hack. But I tested it with every version of Android we support (2.3.6, 3.x, 4.x up to 4.4.3) and it just works. It uses knowledge of an internal mechanism which is subject to change even at the next internal revision, but it seems that the code I examined has been stable for a while. And maybe, by the time it changes, they will fix the main issue (use a select function to control presence of a card) as well!

# Sunday, 10 August 2014

Smartcard authentication and access control

DISCLAIMER: I am NOT a crypto expert. Of course I use cryptography, authentication protocols, etc.
I understand them (and their cleverness amazes me) and I have even taken 2/3 courses on cryptography during my Master and my PhD. But the first lesson you ought to learn when you are working on authentication, and on cryptography in particular, is: you are not, and the courses will not make you, an expert. And so you better never, ever design your own crypto algorithm, or your own authentication protocol. Stick with what others have done (and studied for years) and follow best practices. Not blindly, of course, but one matter is understanding what is going on, and another matter entirely is designing them flawlessly.

Last time I mentioned that one of the possible usage scenario for a smart-card is to act as an access and "presence" token. You can think about them as a read-only ID.
Well, you usually go further than that: writing on cards has advantages (you can update them with new information, for example, or "stamp" them when you see them, so that officials can  check that you have validated it and therefore you are authorized, which is a necessity if you do not have gates at any access point); however for the sake of discussion, we can think about them as read-only IDs.

Couplers, i.e. card readers/writers, will recognize the card (and optionally "stamp" it), and record your passage (via LAN or WiFi or UMTS).

Each card has a unique identifier (UID). This ID is hard-wired when the card is produced. Think about it as the MAC address of a network card.
The UID is always transmitted in the initial phases of the communication between the card and the coupler. Many applications just stop there: read the UID, whitelist them, let them through if they are on the white list.

Does it work? Sure. Is it a good approach? No, absolutely NOT. And I was (and I still am) amazed at how many contactless systems just stop there. These systems are not secure, at all. You just have to read the UID (any NFC reader can do that, you just need -for example- to touch the card with an Android phone) and then configure a device (an NFC phone, or a programmable card) with the same UID and... presto! You "cloned" the card.
You didn't, of course: these cards are not clonable, not without a (very) substantial effort, time, and expensive machinery.

The UID is just not designed for this purpose. It is used to identify a card, not to protect against cloning.
Let's remind us which are the typical requirement in this scenario. You want to:
  1. identify each user uniquely
  2. prevent the user from "faking" a card - either a non-existing one, or a clone of another user
  3. let each user read his own card content, if they want to, but not the content of other cards,
  4. let the officials (security guards?) read the content of the card,
  5. let only the authorized couplers write the card: nobody else can do that (or the officials could read "fake" validations, i.e. records not produced by authorized couplers, and which the systems knows nothing about - if you can do that, you can pass a casual inspection unpunished).

For doing this, many cards (in our case: Mifare Desfire EV1 cards) provide some mechanism out of the box that already cover most of the points. Furthermore, there are a couple of best practices that let us cover all the points.

Desfire EV1 cards use T-DES keys to provide security and authentication. These cards are essentially file-storage cards, with files organized in a one level file system. The top "directories" of this file system are called applications.

The card, every application, and every file can have three keys each. The keys will be used for access control, and can be viewed as the read-key, write-key, and modify-key.
So, if I authenticate with the card using KeyA, and KeyA is the read-key for App1/File1, I can open and read the content of that file, but not write it.

Keys are symmetric, and are stored on the card and on the coupler (the RFID reader/writer device).
Cards are considered a secure storage element, as keys can NOT be read from cards. There is no way to retrieve them, and cards are engineered to be tamper resistant. Most of them even protected from attempts to directly read their content using differential power analysis.

Keys are stored on couplers using essentially the same technique: they are not stored on disk or memory, but inside a Secure Element (SE) (sometimes called SAM). It is essentially the same piece of silicon and plastic you find in the smart-cards or in your phone SIM card (the have the same physical format as well), and it is in fact a smart-card, with the same tamper-resistence properties. So, even if someone steals one of your couplers, he/she still hasn't got the key.

Obviously, only some couplers have the write key. They are also the only devices using a Secure Element: readers (couplers with only a read key) do not have them, for cost reasons.

How do you authenticate with the card? This is all covered by the smart-card protocol: in the case of Desfire cards it is a basically a challenge-response where you "prove" to each other that you know the key. You to that by generating a random number, encrypt it, and send it to the other party. The other party decrypts it, generate another random number and send both of them back, encypted. The first party decrypts it, see that its number is included (good, the other party has the correct key), and sends back the other number, encrypted. So the other party knows we have the key as well (we can encrypt and decrypt).

Previously I mentioned some best practices you ought to add to the basic framework. These are commonly accepted as standard and easy to implement: key differentiation and blacklisting.
Each card you issue should hold a different set of keys. This can seem like a problem, if you have, like us, half a million cards (and therefore 1 million keys).
Wow.. how can we store in an efficient way millions of keys? And check all of them when a card is presented to our couplers?
Well.. we do not store them. We derive them.

The UID of the card is used with a master key to derive a "personalized" key, using a one-way hash.
So even if a card is defective and compromised, we have lost its set of keys, nothing more.
This has several advantages:
  • each card holds different keys, so if one is compromised, you just have to throw away (blacklist) that card
  • (assuming you have chosen a proper hash algorithm) it is impossible to reconstruct the master key from the derived key
  • you can just store the master keys on you "official" devices, in a secure way (i.e. in the SAMs) and they will be able to read and write any card (by deriving the right key on the fly)
  • you can hand off to the user their own, "personalized" key, and they will be able to read their own card. But not the card of their friends (or spouse).

By using what the cards manufacturer offers and two little (and simple to implement) best practices, you have a very good level of security.
We added a little more, just to be on the safe side, but these best practices already are a very good point. I am amazed at how many companies fail to realize the issues and threats they are facing with the extremely naive UID-only implementation.
But we still have sites that store passwords in clear text, or that force the user to a numeric PIN of 5 digits, or to 8-char passwords.. so I should know better by now :)

# Saturday, 08 July 2006

One thing I really don't like...

...is when search engines (like Google, but also MSN) "automatically" sense what language I want to use. I live in Italy, therefore Google thinks I want results in Italian, or from Italian web pages. Well, that's NOT true. I want results and pages in English (like I want to type on an English keyboard and I also want my OS and programs to be in English). Especially if I type google.com instead of google.it. Also, when I search for development-related answers, as a result of this behaviour, I get really poor search result.

Speaking of translations: one of the worst things I have ever seen? Compiler errors (C++) translated into Italian. Even worse, VBA keywords used to be translated into Italian in some old release of Office. THAT was really crazy! :)

# Thursday, 06 July 2006

Windows Vista and Office 2007 Beta 2 (2)

Over all, I liked Office 2007 very much. The only thing I could not test was weblog posting, since dasBlog is not currently supported in the Beta version (it will be in the final version). I surely hope it will work well!

Instead, I have different feelings wrt Windows Vista. I followed Longhorn closely from the first PDC (2003?) and I was really looking forward to see it. Remember the “native kernel + managed subsystems” part? The three pillars, Indigo, Avalon and WinFS? Well, Indigo and Avalon are great but will be part of .NET 3.0, also available for XP, and WinFS is dead

So, what’s the point of Windows Vista? The three pillars gone, it remains the new UI, the improvements to the kernel and window manager, and the improved security model.

Running as a simple User and having to use tedious runas commands to do very common tasks on my notebook (such as changing IP, power profile or directory permissions) I thought the new LUA model of Vista will be great for me. The default user is still marked as “Administrator”, but I think (hope?) it is a simple User account under disguise, and when performing security related operations (i.e. clicking on buttons with the little shield) the security token is upgraded and substituted with one of the Administrator group, if the user grant the permit.
This is my first complaint: why they did that? Be clear, use default account from the Users group and simply ask for an Administrator password before running administrative programs, or on their first security related operation; then make the admin take ownership of the whole program. Surely, it is safest to ask every time if a program can do this or that…or not? People gets bored very easily, and do not always read what it is written on dialog boxes. Normal users almost never do that, they only try to get rid of that annoying (or for someone scary) box that prevent them “using my computer”. Despite this, the new LUA is still better than the previous situation.

The new window manager-UI instead is great. And I’m not only speaking about all the eye-candy (transparencies, the new Flip 3D I already love, the shiny new icons, the sidebar etc.) but also about usability. I love the new explorer UI and the new Open and Save dialog boxes. Finally we went a step further, stripping away the file menu where is no longer useful (like in Office 2007, where it “evolved” into the ribbon) or necessary (like in Explorer and Internet Explorer, where it is… no more!). The click-able locations on the address bar, the search box in the toolbar (yes! No more stupid search dogs…) and the new copy/move progress dialogs are some things I have waited for, and they are really great. And the Sidebar is both useful and beautiful to see (only one complaint: why is it not easy to hide and show it? Maybe with a F-something key, a-la Exposè?).
On the negative side, I have found the new UI very poorly configurable and customizable: If you chose Aero, you can’t even change colors. Very little can be done, but maybe this is the price to pay for a mainstream and “standardized” OS.

Finally, I know this is only a Beta, but I had a LOT of problems installing programs: cygwin does not work, it is impossible to register file extensions for some programs (7-zip comes into my mind), other programs crash without reason. Even SQL Server 2005 needs a patch to work correctly! There is still much work to be done, and it passed a lot of time. Maybe Mini is right, and the Windows team need to change direction.
The course the event took really disappoints me. Vista is great, but not so great and not so radically different to justify for me the switch from XP (a 5 year old OS!). I love .NET, and the managed and secure world, and I’m with Gary McGraw when he says that Vista is a missed opportunity for a new, modern and secure OS; the three-years-ago Longhorn still looks better to me than the actual Vista. I’ll have to wait for Singularity… :)

# Tuesday, 04 July 2006

Windows Vista and Office 2007 Beta 2

I could not resist, and installed both :) I am very pleased with Office 2007. I think is the best release of Office I have ever used! The ribbon is great, although there are some command that are “misplaced”, but I think this is my opinion and it is based on the use I do of Office (Word, in this case). Everybody uses Word or Excel in its own way, so the Office team had done a great work examining usage patterns, correlations between command usages and position, and so on. Just read Jensen Harris blog; as an example, I found particularly amusing his post on “command lingering”.

I used Word, Excel and PowerPoint for about a month now, and I have to say I really like the ribbon, the mini-toolbar, even the Office “Start” button I initially feared so much. And it looks great on both XP and Vista too! I have found myself using and discovering more and more office commands, that I am sure where there even before Office 2007 but were essentially “ghost commands”, buried into menus and not readily accessible.  A really great piece of software, even in the Beta version.

# Wednesday, 26 April 2006

Notable news

Some funny / useful things I found browsing last week:
  • From Brad Adams, .NET Framework 2.0 Poster
    It is the replication of the version 1.1 I found shipped with Visual Studio 2003; unfortunaltely the same poster wasn't in my Visual Studio 2005 box. Great!
  • selk'bag
    The SELK’BAG is a sleeping bag you wear. The entire bag fits like a glove. You can walk with it, sleep in a sit position.. I think it would be great to have one of these for my work place in summer: the ice-conditioning system makes our offices freezing cold!
  • TiwyFeeds
     "Take It With You" Feed Reader: the idea is great, the realization is poor..

    But I'd definitely love a feed reader 'a-la' gmail, where you can store all your feeds and news for an unlimited amount of time, with a searchable interface!
  • Krugle
    A search engine for developers: personally I use google + MSN desktop search (for code on my machine). But a tool tailored to developers is surely great! I'd like to have Visual Studio search and navigation facilities in a desktop serch application...


# Wednesday, 05 April 2006

I am Windows 2000

This is ironic...

You are Windows 2000 SP3.  You're a steady and reliable friend.  People think you're all business, but with your recent therapy you've become a little more playful.
Which OS are You?

But I like Win2000 very much!
# Friday, 03 February 2006

Internet Explorer 7

Finally Microsoft released Internet Explorer 7 Beta 2 for non-Vista users! I downloaded and installed it immediately. I miss IE, it was surely faster and more comfortable to use than Firefox, but during the last year it surely become technologically obsolete.

So, what I like about Internet Explorer 7?

  • New toolbars, no menubar: it's great. It's neat, simple to use, leaves a lot of space for the pages. Who needs a complete File, Edit, View, Window, Help menu in a web browser? Finally someone understood that! A+!
  • Tabbed browsing: I do NOT use it in Firefox, because I always get lost in tabs. I think its a nice feature, and IE surely makes it simple to use and seamlessly integrated. QuickTabs are great, although Firefox have a similar extension (foXpose).
    (BTW, I don't really get tabs: why? They are even more confusing for inexperienced users, because the system already has another tab-bar, the application bar)
  • Search box (finally!). It is possible to add engines, but the number is still scarce: no Webster, no Wikipedia... MSDN is here, fortunately.
  • Search also from the address bar: Mozilla had it, why did they remove it from Firefox I will never get. And it uses the default search engine, too.
  • (Unchanged form IE6, but impossible to have with Firefox) Opening a new Tab/ Window: you can choose to open it showing exactly the same page you are watching now. In Firefox I am forced to open a blank page, then copy/paste the address between address bars.

And what is still behind Firefox? Or, what I hate of IE7

  • First, the most horrible think: ALIASED FONTS.They look terrible on my monitor, why if I disabled them in the control panel IE must impose its will and use aliased fonts? They are orrible to see in my opinion, make me wonder if there is fog in the room..
  • Download boxes: one box for download is surely better than Firefox download manager (awful, IMO) but fortunately firefox have a great extension, Download statusbar

  • Extension: if I don't like something in Firefox, there is an extension to modify it. I love extensions, and if IE7 will be programmable using .NET (maybe JScript.NET) I will be the first to use it and to write a lot of extensions.
  • One of the features I love in Firefox and Visual Studio: incremental search. Guys, you made such a nice work with the tool-bars and the notifications (the yellow ribbon), why ruin the user experience prompting a search box? (or a download dialog, btw)
  • On the same line: download files in the right place from the beginning. Why put them in the cache and then move them? I incidentally clicked "cancel" during this move operation (I was typing) and I had to download the file again.

Well, at the end of the list.. I still prefer Firefox. The only hope of IE is to be programmable, and then it will be mine.