# Tuesday, 31 December 2013

First version of MiniPoint released!

Last Saturday, I installed the first test version of MiniPoint in a production environment. All went well (we spent more time waiting for SQL Server 2012 to finish installation than anything else). 

I had only some minor hichups: figuring out how to deploy without VS/msbuild/WebPI (i.e. manually), and how to configure correctly IIS 7 (the production server uses Windows 2008) for .NET 4.5 / MVC 4. 

I solved both issues thanks to StackOverflow :)


What is MiniPoint?

MiniPoint is a MVC4/AngularJS web application that allows you to define:

  • a set of related data/metrics you want to record;
  • the process needed to collect and save them;
  • the people who will need to collaborate to get, read, validate and update those data.

The application will replace an existing Sharepoint 2010 solution which was fulfilling the same role, but that was too heavy (it required its own, beefy server), not so flexible and much, much more difficult to modify and extend. MiniPoint, instead, was designed with the goal to make it easy to modify and extend every bit of the process.

MiniPoint is build around four concepts: Schemas, Views, Lists and Workflows.

Minipoint initial page. The design, based on Twitter Bootstrap 2.3.2, is intentionally simplistic and lean: end-users (the one who will use the workflows) are not computer experts.


You can think of a schema as the data structure needed to hold all the information needed for the process, or better all the information that will be collected and saved during the process. It is like a database schema, or like the header rows in an Excel table. The parallel with Excel here is not a casual one: prior to the Sharepoint solution, end-users used Excel files to keep track of the information. For example, they had an Excel worksheet, with fixed columns, for customers calls; at each call they had to fill in a bunch of columns and save the file on a network share.

A list is an instance of the schema: the empty rows in the Excel worksheet. Each row is created using the Schema as a mould, and filled little by little by the process. 

Let me explain through a simple example: the submission of an issue.

We can simplify the process by assuming that the issue is submitted, triaged (rejected or accepted), assigned to someone, fixed, verified. If the verification step fails, we go back and re-work on it.

It's a 5 step process, with two branch points. I suppose this sounds familiar to everyone who used a bug-reporting software :) even if this very process was created for a customer which have nothing to do with software! Processes like these are very common in companies of all sizes, as there are many occasions in which it is necessary to keep track of progress and history (the small company for which the software was initially concieved wanted to record at least 5 of them).


Schemas (and lists)

For the above example process, we need to collect 

  • The issue
  • Its category
  • Who submitted it (who will also be responsible of checking the outcome, in our simple scenario)
  • Who will work on it
  • The verification outcome (acceptance and comments)

So our schema is composed by six fields:

  • Issue (Text)
  • Category (Enumeration)
  • Submitter (User)
  • Assigned_to (User)
  • Status (Enumeration)
  • Comments (Text)

In parenthesis I have put the data types for the various steps. MiniPoint supports different data types (more on this in a following post), and use the data type to treat, save and render the field in the right way.


The definition of a schema. Working on schemas is very quick: adding, removing, modifying a filed is done client-side using AngularJS, for a fast and responsive UI

Views

In any complex enough process, all the needed data and "status" is not readily available, but needs to be collected over time, by different actors (this is usually the reason there is a process in the first place). Therefore, not all fields can or should be filled from the beginning! At each step of the process, some fields will be entered, some will be updated, others will be visible but will be not modifiable anymore. 

For this, MiniPoint let you create different views on a schema

A view is a subset of fields, each decorated with some additional attributes. Attributes indicate how the data needs to be presented to the user: whether the field will be visible (and when, with conditions over other fields), required, readable or writable... 


Definition and edit of a view element. Notice how you can specify visibility (and if an element is required) as a boolean expression over other fields in the same schema.



What if you figure out, while creating a new view, that you need another field in the schema? Shortcuts to manipulate related elements are spread through MiniPoint

Workflow

When you have the views, you need to "stitch them together": you need to define in which order they will be presented to the end-user, who will have right to access them, and how you will route users through different views based on what they entered so far.

MiniPoint will then associate views to workflow steps; steps can be executed in order, or (based on conditions over variable and field values) you can have branches.

The workflow is described using a small and simple language, with statements for steps (display of views), report generation, branch (expressed as simple, fixed if (condition) goto step else goto step - just the text equivalent of a "conditional" diamond in flowcharts)

The workflow is a reactive program: it will execute one step ofter the other, and then it will suspend itself when it needs to wait (typically, for user input at steps, or for a pre-defined delay). Then, it will resume itself to react to external input: a delay expired, or a user completed one step.

For this iteration of MiniPoint, the workflow text is "translated" (or, compiles into) a .NET Workflow Foundation 4.0 series of activities, and runs using the a self-hosted .NET WorkflowApplication. I plan to replace WF with my own workflow engine, as it is simpler, leaner and easier to port to different platforms (for example, WF out-of-the-box only supports SQL Server storage). Another reason to switch to my own workflow engine is to have better integration with ASP.NET async; even if there are some nice examples on how to combine WF and ASP.NET MVC,  hosting reliably, in an efficient (asynchronous) way the .NET WorkflowApplication proved to be a bit tricky. The support for async, long-running events in ASP.NET could be dangerous (if not handled correctly), and it is still evolving (it changed in an important way, and improved, in .NET 4.5, but it still need some care).

The current code works quite well, but I am not entirely pleased with the result, as it looks more complex than it should be. For now, however, WF works smoothly enough.

The little "workflow language" was created together with the people who will author and modify the business process; it needed to be flexible and lightweight enough to let them change the process easily, but powerful enough to actually express what they needed to model. It is, like I mentioned, a flowchart, with the addition of a global, shared state (variables). 

The language is textual and simple enough; a (javascript based) graphical UI is in the works too.


The workflow language. Upon Save, the code is complied to check it for syntax errors; errors are displayed directly in the UI to help find and fix them.


The list of running workflows. Only steps to which the user has access are shown; plus, it is possible to filter, reorder and group them.


A the UI shown by the workflow while executing a step; the fields in the view are rendered using appropriate controls, base on the data type and the attributes set during the creation of the view.

Permissions

Another required improvement over the Sharepoint solution was to provide finer control over read/modify and general access control. I solved the issue by borrowing some ideas from the NT access control model: every "securable" element (schema, views, lists, even single documents) has an ACL (Access Control List) attached. Each item in the list is a tuple (role, access), where role is either a username or a group and access is a list of allowed operations (read, write, execute).


The UI for setting and assigning permissions to the various defined elements

In this way, it is possible to statically assign some permissions to users or groups. It is also possible to dynamically assign permission through the workflow code, with specific instructions that give permission to a dynamically inserted user. For example, you may want to assign an issue to a particular group or user, and let only that user or group access the following steps.