# Saturday, 30 July 2016

Pumpink: a .NET "container"

More or less 20 months ago (gosh, time flies!) I started a side-project for a very famous company (StackExchange).
StackExchange had just launched, a few months before, a new feature on the main site of their network (StackOverflow).
This feature is called "Code Snippets", and it allows you to embed some sample HTML + JS code in a Question, or an Answer, and let the visitors of the page run it.
Of course, being JS it would run inside the browser, and with few focused precautions it can be made safe for both servers and clients (you do not want to leave an attack vector open on your servers, but you also don't want your visitors to be attacked/exploited as well!)

More details on how they implemented and safeguarded it can be found on their meta.stackoverflow.com site.

The feature got a lot of attention, and of course there where requests to extend it to other languages.
I was one of those that wanted more languages added, but I understood that JS was a very particular case.
Snippets in any other language would have meant an entirely different approach and an entirely different scale of complexity.

In October 2014 I visited NYC; before my visit I got in touch with David Fullerton, the "big boss" of SO development team. We were in touch since my previous "adventure", a few years before, when I interviewed for a position on their Q&A team. We discussed briefly about my past interview, and then he asked me a very interesting question: what would I add to StackOverflow? C# snippets immediately come to my mind.

We discussed briefly about it, drafted up some ideas, added requirements in the process (discarding most of the ideas) and finally David asked if I would like to try it out, as an Open Source experiment sponsored buy StackExchange.

... well, of course! Fun and challenging software, exchanging ideas with some of the most brilliant devs in the .NET ecosystem, and I get paid too! :)

So "Pumpink" was born. If you are curious, you can find it on my GitHub. It already contains a rather in-depth analysis about the structure of the project, and how it works.

Or, if you want to know "why?" instead of simply "how?", you can wait for the upcoming blog posts, in which I will detail some of the choices, problems, headaches that shaped the problem.

# Monday, 06 April 2015

jQuery UI datepicker: being notified of every date change

Recently, I was fighting with a Primefaces component: calendar. I was using it with my keyboard, and I immediately noticed that the ajax update that keeps the control synchronized with the bean was not working. Selecting a date with the mouse was fine, editing and pressing "Enter" was OK, but changing it and tabbing or clicking away was not.

Curious, I dug in and found that primefaces uses the standart jQuery UI datepicker, and wires itself to its standard onSelect event. But this event is fired only when you explicitly select a date, not when a date changes (through the keyboard, or code).

So, I decided to write a quick js to call onSelect when a date is updated. Here it is: I also keep track of the previous date, to avoid double notifications. And it works very well for us!
# Monday, 15 December 2014

Strangest bug of the day

System.TypeLoadException:
Could not load type 'System.Diagnostic.Debug'
from assembly 'MyAssembly'

Uhm... Hello?
Why are you looking for 'System.Diagnostic.Debug' inside my assembly? Go look inside System!

To be fair, I was fiddling with the CLR Hosting APIs, in particular IHostAssemblyStore.
Using this interface, you can override the CLR assemby probing logic, and load you own assemblies from wherever you want.
This is particularly helpful if you are using your own AppDomainManager, and you want to load it inside your sandboxed domain, which you carefully configured to disallow loading of any external assembly.
If you try to do that, binding fails, obviously. And since the AppDomainManager is created and loaded by the unmanaged portion of the CLR, you don't get a chance to use AssemblyResolve (tried, didn't work).

But that does not really justify this behavior: the Fusion log viewer was reporting no errors, and the exception was wrong (or, at least, not one of the usual exceptions you should look for when debugging binging failures).

There is a tiny parameter, called pAssemblyId. The docs are quite clear: it is used internally to see if that assembly was already loaded.
If it is, do not try to load it again. Sounds like a good optimization BUT:

The docs don't say it, but if you pass back "0", something very peculiar happens.
In my case, I had the Jitter looking for types inside my assembly. Others had different issues.

That was definitely the strangest bug of the day :)

# Saturday, 16 August 2014

AngularJS and ASP.NET MVC validation

Not really a blog post, just a collection of links and articles I used when I was writing validation code for Angular with ASP.NET MVC.
More a "reminder to self" than anything else, as they are a bit outdated (they refer to ASP.NET MVC version 4, mostly) but they could be helpful to others as well!


# Friday, 15 August 2014

Android NFC service and "thin client": one problem, and one hack

Lately (in the last year or so), Android work intensified at my company. So, I finally took the time to study it in depth and I discovered how MUCH Android differs from what I was expecting. It really starts to make sense when you dig under the cover. And you start to discover how much better your apps behave when you are using the SDK the way it should be used (and you also start to pick up defects in other apps and say "Ha! You did that! Gotcha!).
But this is the topic for another post... :)

Today I want to concentrate on an issue I was experiencing using the NFC framework in Android to read our contactless cards.
Using the NFC framework as a general purpose card reader is a bit on the "stretchy" side: the framework, after all, is mainly there to read Ndef tags, which have a precise structure. Fortunately, Android allows you do go deeper, and interact directly with a card using a "transceive" method.

In this way, you can send commands and data to the card (in the form of a byte[]) and receive a response from the card (again, in the form of a byte[]).
So far, so good: this means that we can read our Mifare Desfire cards, using the Desfire authentication and our keys.
I implemented the commands to authenticate a card, select an application (i.e. a protected directory inside the card memory) and read the data.

All is working well, but.. you have to store your key on the phone, and the storage on your phone is not secure.
In theory, every application has a local storage that cannot be read by other applications. In practice, you just have to have root access to your phone (which is mighty easy with Android handsets) and you are done.

This is not a particular problem for some scenarios (e.g. if you provide an app that uses the user differentiated key, so that the user can read his own card), but it is a problem when you need to read multiple cards, and therefore to use the master key.

Suppose you are a third-party company. You are my friend, and you want to provide a discount for my subscribers (people that have my smart-card).
How can you check that the card is real, and that the card is not expired? Easy, you authenticate with the card and read its content: the expiration date is written right there.
But I do not trust you enough to let you have my read keys!

Maybe you even want to top up my card with "reward points": if my users buy something from you, they will have discount on my services. Super cool!
But I will not let you have my write keys.. that's out of question!

Sure, you can read just the UID, and use that to look up user info on my web-service. And use the same service to POST reward points. But my network is sparsely connected, and it might take long before a card is used on one of my terminals and I can update them.
And we have seen that a UID can be faked..
So?

The answer is "thin-client". You use your NFC phone as an antenna, nothing more. What you read from the card is sent as a (hex-encoded) string to a web service. The web service contains the logic and data to interpret the request and prepare the right response. The response is sent back to the phone, and then transmitted to the card.

You can authenticate with the card, but your keys are safely stored away on your server and they never transit on the the phone!
The phone does not even see the personalized key, so the user is safe against cloning.
I build a prototype, and it worked great on our WiFi network.
They I tried to use it on a cellular network and it failed (almost) regularly. Why?

My suspect was that after a (very short) while the card was reset.
The answer I was getting back from the card was something like "operation not supported in this state". It was like somehow the card forgot that we were in the middle of an authentication challenge-response before the protocol was over.
I decided to investigate, to see if my suspicion was confirmed.
Fortunately, Android is OSS and source code is available! So I dug into the Android source code, looking for clues in the NFC implementation.

Android implements NFC using a mix of libraries and processes; most of the NFC stack is native, and managed by the OS. Then, there is a Service (provided by the OS) that handle communication with the native NFC stack. And some client-side classes you can use inside your application, which will communicate with the Service, hiding it from you.
I started to dig into the source by following a "tranceive" call.

On the application side, you receive an Intent when a card is presented to the reader. Inside the intent payload there is a class derived from BasicTagTechnology; in our case, we use a ISO-A compatible card, so we get a IsoDep object.

The most important method of this class is, as I mentioned, tranceive:

IsoDep.transceive
BasicTagTechnology.transceive

The method inside is just a thin wrapper for remote invocation to a service, which is the NfcService or NfcApplication (the name has changed between Android releases:

Tag.getTagService().transceive(mTag.getServiceHandle(), data, raw)

class Tag ...

    public INfcTag getTagService() {
        return mTagService;
    }
 
INfcTag is an aidl interface, which is used to forward data and commands to NfcService.
We can follow the transceive implementation inside NfcService:

public TransceiveResult transceive(int nativeHandle, byte[] data, boolean raw)   
 
 tag = (TagEndpoint) findObject(nativeHandle);
 response = tag.transceive(data, raw, targetLost);
 ...
 Object findObject(int key) {
        synchronized (this) {
            Object device = mObjectMap.get(key);
            if (device == null) {
                Log.w(TAG, "Handle not found");
            }
            return device;
        }
    }
  ...

So, there is another "Tag" class inside the service; all known (in range) tags are held by the NfcService class in a map.
This "Tag" is named NativeNfcTag:
    
public class NativeNfcTag implements TagEndpoint
   ...
   private native byte[] doTransceive(byte[] data);
   public synchronized byte[] transceive(byte[] data) {
      if (mWatchdog != null) {
         mWatchdog.reset();
      }
      return doTransceive(data);
   }
   ...

The implementation of doTransceive is native, and it varies from a card tech to another.
We have found the end of the flow. Have we also found any clue about the card reset?

The answer is there, inside NativeNfcTag. You should have notice the "mWatchdog.reset()" statemente inside doConnect. What is mWatchdog?

private PresenceCheckWatchdog mWatchdog;
    class PresenceCheckWatchdog extends Thread {

        private int watchdogTimeout = 125;

        ...

        @Override
        public synchronized void run() {
            if (DBG) Log.d(TAG, "Starting background presence check");
            while (isPresent && !isStopped) {
                try {
                    if (!isPaused) {
                        doCheck = true;
                    }
                    this.wait(watchdogTimeout);
                    if (doCheck) {
                        isPresent = doPresenceCheck();
                    } else {
                        // 1) We are paused, waiting for unpause
                        // 2) We just unpaused, do pres check in next iteration
                        //       (after watchdogTimeout ms sleep)
                        // 3) We just set the timeout, wait for this timeout
                        //       to expire once first.
                        // 4) We just stopped, exit loop anyway
                    }
                } catch (InterruptedException e) {
                    // Activity detected, loop
                }
            }
            // Restart the polling loop

            Log.d(TAG, "Tag lost, restarting polling loop");
            doDisconnect();
            if (DBG) Log.d(TAG, "Stopping background presence check");
        }
    }

    
    
The "watchdog" is a thread that at short intervals (125ms) checks if the card is still in range, using the "doPresenceCheck()" function. Which is native, and card-dependent.

The function could be therefore an innocuous instruction (a no-op), or a new select that will reset the card to its not-authenticated state.
Guess which one is for Desfire cards?

So, if the watchdog is not reset periodically by transmitting something to the card, a presence check will be triggered and the card will be selected again, resetting the authentication process. While you are still waiting for the cellular network to answer (125ms is a short time on 3G).

I started to think on ways to work around it, from suspending the thread (inside another process - the service - in Android? Root necessary), to set the timeout (by invoking a method on NativeNfcTag using reflection... again, another process was out of my reach) to substitute the code for "doPresenceCheck()" (which you can do with things like Xposed, but.. that will require Root access too).

You just cannot access anything inside another process in Android, if you don't have root access. Which is usually a very good thing indeed, but it getting in our way in this case.
But what about our process? Sure, we can do almost anything inside it... but how can it do any good?

Well, there is a function inside NativeNfcCard which we can use. This function is "exposed" from "Tag" (the non-public class used at "client" side, see above), but not by BasicTagTechnology.
So we cannot call it directly (like transceive), but from the Tag class onwards it follows the same flow as transceive. This function is "connect":

class Tag {
   ...
   public int connect(int nativeHandle, int technology)
   public synchronized int connectWithStatus(int technology)
   ...
}

If we examine the source code of "doConnect" on the other side (its implementation inside NativeNfcCard) we can see that this function will reset the watchdog too (like transceive). Moreover, we can turn "connect" into a no-op:
private native boolean doConnect(int handle);
    public synchronized boolean connect(int technology) {
        if (mWatchdog != null) {
            mWatchdog.pause();
        }
        boolean isSuccess = false;
        for (int i = 0; i < mTechList.length; i++) {
            if (mTechList[i] == technology) {
                // Get the handle and connect, if not already connected
                if (mConnectedTechnology != i) {
                    ...
                } else {
                    isSuccess = true; // Already connect to this tech
                }
                break;
If the technology we specify is the same one we are already using, or if it is a non-existing technology, the function will do nothing.

We can just grab the Tag class inside our code, call connect on our side (using reflection, as it is not exposed by the API), and wait for it to forward the command to the service, resetting the watchdog. Do this regularly, and we can "buy" as much time as we want to complete our authentication protocol!

This is obviously a hack. But I tested it with every version of Android we support (2.3.6, 3.x, 4.x up to 4.4.3) and it just works. It uses knowledge of an internal mechanism which is subject to change even at the next internal revision, but it seems that the code I examined has been stable for a while. And maybe, by the time it changes, they will fix the main issue (use a select function to control presence of a card) as well!