Deserializing JSON with c#

Foreword

This tutorial details how to deserialize* a JSON “string” into a c# object using JSON.net by Newtonsoft. It’s a stand alone tutorial, but follows logically from Consuming a REST API from c#.

* It pains me to use a ‘z’ here but I need to keep the US audiences and search engines happy!

If you prefer video to the written word, you can find the accompanying YouTube video here:

 

User Story

As a developer
I want to parse, (deserialize), a JSON string
So that I can work with c# objects

Acceptance Criteria

By the end of this tutorial you should be able to:

  • Understand what JSON is and how it’s structured
  • Deserialize JSON Strings

Ingredients

  • Windows PC (to run Visual Studio)
  • Visual Studio (I use the Community Edition)
  • JSON Editor Online @ http://jsoneditoronline.org/ (you can use an alternative I just like this one)
  • 20 Minutes of your time

User Interface Design

Using Balsamiq the following wire frame emerged and helped clarify my thinking, it’s pretty simple really… It takes and input, input is processed by clicking the button and the output is shown in the bottom plane. There’s a 2nd button that clears the output window.

A good wire frame should speak for its self though, so here you go:

VP-1 Mock Up

Wire Frame – Realized

So here’s what the wire frame translates to in the real world, I’ve place the form element names there if you’re wanting to use the same naming convention as me, (not sure why you would my naming convention is pretty rubbish!):

2017-07-19_21-16-08

Point to note, both textboxes are set to “Multiline: True” and with vertical scroll bars…

“Wire Up” The Buttons

Ok, so let’s do some simple stuff first, double click each button on the form designer to invoke their click event handlers.

Into the “cmdClear” button’s handler place the very simple code:

private void cmdClear_Click(object sender, EventArgs e)
{
  txtDebugOutput.Text = string.Empty;
}

If you don't understand what that's doing then you may struggle with the rest of the tutorial, so moving on!

I also, (as in previous tutorials), like to add my little "debugOutput" function that takes a string, displays it on a multi-line text box and "scrolls" to the next line, so further text can be displayed. So inside your Form code, add the following function, (it's not mandatory for the actual parsing of JSON, but it just looks nice):

private void debugOutput(string strDebugText)
{
  try
  {
    System.Diagnostics.Debug.Write(strDebugText + Environment.NewLine);
    txtDebugOutput.Text = txtDebugOutput.Text + strDebugText + Environment.NewLine;
    txtDebugOutput.SelectionStart = txtDebugOutput.TextLength;
    txtDebugOutput.ScrollToCaret();
  }
  catch(Exception ex)
  {
    System.Diagnostics.Debug.Write(ex.Message.ToString() + Environment.NewLine);
  }
}

Let's test the function by adding the following code to the cmdDeserialise Click event handler. Test it by typing something in txtInput, and clicking cmdDeserialise...

private void cmdDeserialise_Click(object sender, EventArgs e)
{
  debugOutput(txtInput.Text);
}

Yeah - amazing right?

giphy-downsized.gif

So what is JSON?

I'm going to guess if you're reading this you have some idea... So I don't really want to go over in vast detail what it is here as there are plenty of great written resources already, (check out the Wikipedia entry here).

What I do want to do though is labor some points that will come in useful when we start deserializing the raw JSON strings...

  • Stands for: "JavaScript Object Notation"
  • Open format used for the transmission of "object" data, (primarily), over the web.
  • It consists of attribute-value pairs, (see examples below)
  • A JSON Object can contain other "nested" objects

Anatomy of a Simple JSON Object

simple json

In the above example we have a "Person" object with 4 attributes:

  • firstname
  • lastname
  • age
  • isAlive

With the following respective values:

  • Roger [This is a string data-type and is therefore delineated by double quotes ' " ' ]
  • Moore [Again this is a string and needs double quotes
  • 89 [Number value that does not need quotes]
  • false [Boolean value, again does not need the double quotes]

Paste this JSON into something like jsoneditoronline.org, and you can interrogate its structure some more, (possibly not that useful for such a simple object...)

simplejson in editor

A, (slightly), more complex example

As mentioned in the overview of JSON, an object can contain "nested" objects, observe our person example above with a nested address object:

complexJSON

Here we can see that we have a 5th Person object attribute, address, which does not have a standard value like the others, but in fact contains another object with 3 attributes:

  • streetAddress
  • city
  • postcode

The values of all these attributes contains strings, so no need to labor that point further!

This nesting can continue ad-nauseum...

Again posting this JSON into our online editor yields a slightly more interesting structure:

complexjson in editor

A final example

Onto our last example which this time includes an array of phone number objects:

arrayJSON.png

Note: I removed: "age" and "isAlive" attributes from the person object as well as the "postcode" attribute from the address object purely for brevity.

You'll observe that we added an additional attribute to our Person object, "phonenNumbers", and unlike the "address" attribute it's contains an array of other objects as opposed to just a single nested object.

So What?

So why did I detail these JSON examples, we'll firstly it was to get you familiar with JSON and some of its core constructs, specifically:

  • The start and end of an object: ' { } '
  • Attribute-Value pairs
  • Nested Objects, (or objects as attribute values)
  • Array of Objects

Personally on my JSON travels these constructs are the main one's you'll come across, and as far as an introduction goes, should give you pretty good coverage of most scenarios.

Secondly, we'll be using these examples in deserializing JSON with JSON.Net, so lets's get started on that!

Installing JSON.NET

The best way to install JSON.Net into your Visual Studio project, (i think), is to use the NuGet Package Manager Console. So in your project, do the following:

NuGet

Once in the Package Manager Console, type:

PM> Install-Package newtonsoft.json

See below:

nugert success

You should also be able to see JSON.Net in the list of Project References:

jsonnet references

OK, We're Ready to Code

In order to deserialize a JSON string we're going to use the JsonConvert class from the JSON.Net framework, then use the DesrializeObject method, a link to the Newtonsoft web site can be found HERE, detailing this approach.

Let's be Dynamic

Ok our first example requires the least coding, first create a new method in your form's code called: deserialiseJSON, as follows:

 
private void deserialiseJSON(string strJSON)
{
  try
  {
    // TO DO: Our deserialize code
  }
  catch(Exception ex)
  {
    debugOutput("We had a problem: " + ex.Message.ToString());
  }
}

Now update the click event of our deserialize button:

 
private void cmdDeserialise_Click(object sender, EventArgs e) 
{ 
  deserialiseJSON(txtInput.Text); 
} 

This just means we're almost ready to start deserializing!

Go Back to our deserialiseJSON method, and in our try block, add the following code:

 
private void deserialiseJSON(string strJSON)
{
 try
 {
   var jPerson = JsonConvert.DeserializeObject<dynamic>(strJSON);
   
   debugOutput("Here's our JSON object: " + jPerson.ToString());
   debugOutput("Here's the First Name: " + jPerson.firstname);
 }
 catch(Exception ex)
 {
 debugOutput("We had a problem: " + ex.Message.ToString());
 }
}

So what have we done here?

  • Created a variable (var) called jPerson. This will hold our deserialised JSON object
  • Used the DeserializeObject method of the JsonConvert class
  • Used the "dynamic" object as a place holder for our "type" - more on this below
  • Passed in what ever serialized, "raw", json string is in our first text box

It's the use of dynamic that's most interesting here. This basically tells Visual Studio that we'll have to wait until run time before our object dependencies are resolved. In practice this means we can pass in any valid json string and the JSON/Net frameowrk will interpret and create a deserilaised c# object for us on the fly, (jPerson).

We can then access the attributes of jPerson, and treat it like a regular object.

So let's run our app, and pass in our simple json example:

examppl1

Success!

To Come

Still to write up the use of c# classes and supply those as our type.

 

 

Authenticating to a REST API from c#

Foreword

This tutorial follows on directly from my previous post: Consuming a REST API from c#. You may need to start there if you want to pick up the thread of the coding examples below.

 

A full video of this tutorial can be found here.

 

User Story

As a developer
I want to call a REST API that requires authentication
So that I can consume its “protected” resources

Acceptance Criteria

By the end of this tutorial you should be able to:

  • Authenticate to a REST API (using a c# Windows app), using Basic Authentication
  • Authenticate to a REST API (using a c# Windows app), using NTLM, (Windows), Authentication

Ingredients

For this tutorial you will need the following, (or something similar):

  • Windows PC (I’m running Windows 10)
  • Visual Studio (I’m using the 2017 Community Edition – which is free)
  • Web Browser (I’m using Firefox and Edge)
  • Access to a REST API that uses “Basic” Authentication
    • Don’t worry if you don’t we spin one up as part of this tutorial
  • Access to a REST API that uses “NTLM” Authentication
    • Don’t worry if you don’t we spin one up as part of this tutorial
  • ~40 Minutes of your time

My Lab Set Up

Having listed the ingredients above, here’s a simple schematic of my Lab Set up for this tutorial:

VP-6 REST Authentication My Lab

You’ll notice that I’m using the Firefox plugin called “Live HTTP Headers” this isn’t mandatory for the tutorial but it’s a useful little tool that allows you to see the HTTP Headers sent and received by Firefox. Kind of helps demystify the whole thing.

You’ll also notice that I have a Linux VM running “Jira”. This is absolutely not necessary but I included it as: a) I’ll be using it in a later tutorial, and b) it’s good to try this stuff against a proper production API.

What’s “Jira”?

Jira is a web-based tool from a company called Atlassian, it’s actually quite difficult to describe what it is, but in essence it’s an “Issue Tracker”. Think of it like a big “To Do List” repository for organsations. It’s used a lot by Technical help desks, (to raise fault / support / defect tickets), and software development teams to capture requirements and plan / track the release, (as well as bugs).

I actually use it to plan the tutorials & Youtube videos I’m going to produce, I even knew a guy who used it to plan his wedding!

I’ve included it as is provides a actual REST API interface that I can test against. You can safely ignore it if you want to though!

User Interface Design

Again using Balsamiq I created the following wire frame to help clarify my thinking on what the app is going to do. You’ll notice I’ve provided for:

  • username and
  • password credentials

which will be passed to our REST Class on an as needed basis. I won’t be covering how to  securely store credentials persistently as part of this tutorial. If that’s something of interest though drop me a line!

There’s also 2 radio button groups:

  1. Authentication Type
  2. Technique

Which I thought may be necessary at the start of this project to dynamically switch between, (surprise-surprise!), Authentication Type and Technique. As it transpires they’re probably a bit superfluous, but I’ve left them in for now.

VP-6 REST Authentication Wireframe

Class Diagram

We’re basically just extending upon the Class from the previous post, the updated diagram is shown below:

VP-6 REST Authentication Class DiagramI’ve added 2 new enumerations:

  • authenticationType
  • authenticationTechnique

Think they’re self explanatory, so we’ll move on…

Our constructor and makeRequest method have not changed, at least from a calling perspective, there are obviously some internal changes which is what we’re going to code up below.

We have some new attributes, again I think self-explanatory:

  • authType
  • authTech
  • userName
  • userPassword

Is that a good constructor?

You’ll notice, (when it comes to coding below), that the constructor doesn’t take any arguments and that we set all the class attributes, (e.g. authType etc.), following construction.

There is no real reason, (that I can think of), why this really needs to be the case, so obviously feel free to change that!

 

Coding – Part 1: Self Rolled “Basic” Authentication

Ok ok! We’re here finally at the coding – I’m guessing the bit you’re all interested in! Again to pick up this coding thread you should start here.

I’m going to assume that you’ve updated the UI with the following elements:

VP-6 Form Controls

So basically your UI will look like the following:

VP-6 REST Auth Form UI

Now we’re going to update our RestClient class as follows, (new code from the previous tutorial is in blue):

using System;
using System.IO;
using System.Net;


namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    public enum authenticationType
    {
        Basic,
        NTLM
    }

    public enum autheticationTechnique
    {
        RollYourOwn,
        NetworkCredential
    }

    class RestClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }
        public authenticationType authType { get; set; }
        public autheticationTechnique authTech { get; set; }
        public string userName { get; set; }
        public string userPassword { get; set; }


        public RestClient()
        {
            endPoint = string.Empty;
            httpMethod = httpVerb.GET;
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();

            String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
            request.Headers.Add("Authorization", authType.ToString() + " " + authHeaer);

            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
        }

    }

The really only interesting thing here is the addition of the String "authHeader", and we attach it to our HttpWebRequest object - "request".

What is this header string?

This "self-rolled" header string supports "Basic" Authentication - see the section below. We construct it so that it follows RFC2617 - The HTTP Basic Authentication scheme and pass it with our initial request so that we are authenticated through, (assuming the credentials are correct).

As per RFC2617, the string should be constructed using the following method:

  1. Obtain the user id and password (this will be supplied from our text boxes)
  2. Construct the "user-pass" be concatenating the user id, a single colon (":") character and the password. E.g. "john:abc123"
  3. Encode the "user-pass" into an octet sequence
  4. Encode the the octet sequence using Base64 into a sequence of US-ASCII characters

The spec goes into more detail, you can read it here.

All we need to do now is update out "Click" button event to pass the user name and password, (we can also set the Auth Type and Technique values here although they are not really used at this stage).

The updated click event handler will look something like this, (new code is in blue):

private void cmdGO_Click(object sender, EventArgs e)
        {
            RestClient rClient = new RestClient();
            rClient.endPoint = txtRequestURI.Text;
            rClient.authTech = autheticationTechnique.RollYourOwn;
            rClient.authType = authenticationType.Basic;
            rClient.userName = txtUserName.Text;
            rClient.userPassword = txtPassword.Text;

            debugOutput("Rest Client Created");

            string strResponse = string.Empty;

            strResponse = rClient.makeRequest();

            debugOutput(strResponse);
        }

If all is successful then this should allow you to use a REST API that uses Basic Authentication.

As an example I called my Jira REST API, (I wanted to retrieve the JSON for a single issue with id: wp-1), with the following request:

Note: I did not pass the any credentials over, and so I received the following response:

vp-6 unauthorised

You can see that Jira returned back a 401 response, (unauthorised). More about that later...

Trying again but this time supplying the correct credentials yields success and some JSON related to the issue:

vp-6 authorised

How Basic Authentication Works

There are plenty of resources on the internet that detail how Basic Authentication works, (I have placed a few links in the references section below), but thought I'd give a brief overview here as its relevant to a point I want to make...

The following sequence diagram illustrates a typical request response scenario when the initial request does not have the necessary authentication header:

VP-6 REST Basic Auth Sequence

You'll see that the 2nd request supplies and Authorisation Header as per RFC2617 and we are returned a successful 200 OK response.

This "challenge/response" scenario is important when we come to look at the NetworkCredential class as it relies on this to supply the credentials. I.e. it won't initially supply the credentials until it receives the 1st 401 Unauthorized response along with the Authentication type, in this case "Basic".

If it, (the NetworkCredential Class), does not receive this response, then it may not work correctly. This is actually the scenario I encountered with the Jira api, that being when I used the NetworkCredetial class with the Jira API I could not get it to work. Had I read the Jira API documentation I'd have realised that Atlassian, (the company that make Jira), did not implement their API that way, observe:

vp-6 jira doc

Why do you care?

Good question! The reason I have made a bit of a diversion is 2 fold:

  1. When we come to the NetworkCredential Class below it is useful background
  2. In researching this tutorial I have found many people having a similar issue with the NetworkCredential class, (when in fact it's not really the NetworkCredential class to blame, but API vendors that haven't implement the Basic Authentication standard correctly). Basically, (no pun intended), I don't want you to waste the same amount of time I did.

TIP

As boring as it sounds, READ your API's documentation. I may save you a lot of time!

Set Up Your own API?

The thought occurred to me that perhaps some people don't have access to a test api? Or in my case I wanted to spin up a really stock-standard api so I could do more testing and play with different types of authentication.

I was also confused why the Jira API did not work with NetworkCredential class so wanted to test with another Basic Authentication API I set up myself - as mentioned above had I read the API documentation my confusion would have dissipated!

I therefore decided to insert this section into the tutorial so I:

  • Could Demonstrate using the NetworkCredential Class
  • Play with NTLM, (Windows), authenticaton

How Many Types of Authentication?

There are a plethora of different authentication mechanisms in use, so would have been difficult for me to cover them all. I chose Basic and NTLM in this case.

Set Up IIS

Ok since we're running on Windows and as per my lab set up above, we're going to run a REST API using the Microsoft ASP .NET MVC framework, (don't worry if that doesn't make sense), which will require that we have IIS installed.

What About IIS Express?

We could use the embedded version of IIS, (IIS Express), that starts when you run your ASP.NET project from within Visual Studio, but I've found that doesn't give you the same power and flexibility when it comes to configuring the authentication options.

Step 1 - Open Control Panel
  • Click "Programs"

VP-6 Control Panel

Step 2 - Click: Turn Windows features on or off

vp-6 windows features

Step 3 - Select the IIS Features You Want, (or need!)

vp-6 iis

You'll notice I've placed arrows against the primary artifacts you'll need to run a REST API with authentication...

Click OK and assuming all's well the required components should be installed.

You can check that IIS is working ok by firing up a browser and navigating to "localhost": http://localhost, you should see something like this:

vp-6 iis welcome

Spin Up a Template API

OK cool so IIS is up an running, (hopefully), so all we need to do now is spin up a REST API... Here we go!

Step 1 : New Project
  • Open Visual Studio and select New Project
  • Expand the Visual c# node
  • Select "Web"
  • And Select "ASP .NET Web Application (.NET Framework)
  • Give it a name and hit OK

vp-6 New Project

Step 2: Select the "Web API" Template
  • Select the "Web API" Template
  • Ensure both "MVC" and "Web API" tick boxes are checked
  • DON'T Click OK YET! (We need to set authentication type)

vp-6 aps template.png

Step 3: Click "Change Authentication"

vp-6 change auth

Step 4: Select Windows Authentication
  • Select Windows Authentication (We can change this in IIS Manager later)
  • Click OK

vp-6 windows auth

Click OK once more, (to close out the ASP. NET Template Selection Window)

Your API Project will be created!

Step 5 - Edit the "Index" Method of the "Values" Controller

Don't worry if you don't underand what this is, we can cover it in another tutorial.

  • In the Solution Explorer, Expand the "Controllers" folder
  • Double Click the "ValuesController" Class file - the file should open in the editor
  • Optional: Change the "Value 1" and "Value 2" values in the 1st method to something else.
  • Edit the "Authorize" Directive at the top of the class to include a user and specified account. This will take the form: domain\username.

If you are running outside of a windows Domain, (if you're running a stand alone PC at home this will probably be the case), then the "domain" value is just your PC name.

Your Edited code should look like this:

vp-6 code

Note my PC name is: DESKTOP-H9580B0

My Account is: lesja

Step 6 - Build

I usually just to a build at this stage to make sure there are no errors:

  • Build Menu->Build Solution

vp-6 build

Deploy and Configure Authentication

Ok now we have our test API build - we can now deploy it to IIS:

Step 1: Right Click Project and Publish
  • Locate your "project" in solution explorer, (it should be the 2nd node under the main solution)
  • Right Click and select "Publish..."

vp-6 Publish

Step 2: Select IIS, FTP
  • Select the "IIS, FTP, etc" Option and click "Publish"

vp-6 Publish IIS

Step 3: Select File System Method
  • Select "File System" as the Publish Method in the resulting dialog box
  • Click on the "..." ellipsis for the target location and locate the folder you want to publish to.

vp-6 Publish to.png

Step 4 - Select Publish Destination
  • Create a New Folder if Required
  • Note that using this method the location you select should be on the same server where IIS is installed, (as we're doing all this on our PC it's fine)
  • Select the folder and Click OK

vp-6 Destination.png

  • Click Next

vp-6 next

  • Expand "File Publish Options"
  • Select "Delete all existing files prior to publish"
  • Click "Save"

 

vp-6 Publish settings2

 

  • The Project should be published to your chosen destination if all is well.
  • You should see something similar in your "Output" window:

vp-6 Publish Output

Step 5: Configure Application in IIS Manager
  • Type "IIS" into Cortana (if using windows 10) or just "Search Programs and Files" in older versions of Windows, (from memory you'll have to click the start menu)

vp-6 IIS

  • Internet Information Services (IIS) Manager should be found. (if it's not go back to the step where we installed IIS and ensure that you have the "Management Tools" box selected)

vp-6 IIS Menu

  • Once IIS Manager has started, expand the tree in the left hand plane until you see the "Default Web Site" node, (it looks like a little globe).
  • Right Click and Select "Add Application..."

vp-6 add application

  • Give the application an "Alias". I usually use the same name as the Project
  • Click the "..." ellipsis for the "Physical Path" destination and locate the folder location where you previously published the app, (see steps above).
  • Click OK

vp-6 app location

  • Expand, (if you haven't already), the "Default Web Site" node and you'll see your configured IIS "Application"

vp-6 new app

OK we're nearly there! All this hard work now means that we can configure the "Authentication" type for our app.

Step 6: Select The Applications Authentication Type

Note: You'll repeat this step multiple times as you switch between the 2 authentication types covered in this tutorial:

  • Baisc
  • NTLM (windows)

In IIS Manager, double-click on the "Authentication" Icon:

vp-6 Click Authentication

  1. Click on "Basic Authentication"
  2. Click Enable

vp-6 Set Basic

You should see that "Basic Authentication" is enabled for this application. You may also get a warning about SSL.

vp=6 baisc enabled

The point about SSL is extremely valid, if you're not using SSL, (essentially "https"), then the credentials you're sending over the wire are not secure...

Step 7: Test Site Manually in a Browser First

To just test that the basic site works, go to your favorite browser and navigate to the applications home page, this will be:

http://localhost/<application alias>

E.g. In my case:

http://localhost/tutorialRest

vp-6 Manual Browse to Home

We can now browse to our, (secured), API, which as you may remember formed part of the "Values Controller".

The URL you'll need is:

http://localhost/<application-alias>/api/values

So in my case this would be:

Assuming all is correct you'll get a 401 challenge from the server:

vp-6 401 challenge

As mentioned right at the start, (seems like a long time ago now right!), I installed a plugin to Firefox called "Live HTTP Headers", this allows us to look at, (surprise surprise), the http headers sent to and from IIS. Here's the output for that first request / response as shown above:

vp-6 http header 1

 

 

You'll notice that at this stage we hadn't sent the credentials...

Now following entering, (the correct!), credentials we get the following http header traffic:

vp-6 http header 2

And finally we get the following output in our Browser:

vp-6 http 200

Note: We can repeat step 6 - Select Application Authentication Type but this time for NTLM, (windows), authentication, (remembering to "disable" Basic Authentication first). If you do this and you interrogate the http header you'll see something like the following in the first response leg:

vp-6 http header 4

Step 8 - Test with Our C# Client

Ok, before we begin ensure that the test API has been set to use "Basic Authentication" once again!

  • Fire up our c# Rest Window client and make a first request to the test api, (don't supply any credentials you should see:

vp-6 Rest Client 401

You'll see that we correctly get a 401 response from the server.

  • Now let's enter our correct credentials and try again:

vp-6 Rest Client 200

And we have success!

We've basically called our test API, (that's using Basic Authentication), from our c# client by constructing our own header.

What happens with we switch the authentication type to NTLM, (windows), authentication and try again with our client?

vp-6 ntlm 401

yeah that's right we get a 401 response because we're still using a self-rolled authentication header that's used only by Basic Authentication.

What we need now is the "NetWorkCredential" Class!

Coding - Part 2: NetworkCredential

In our previous example we constructed our authentication header on our own, but we could have used a much simpler method - the NetworkCredential class, (assuming Basic Authentication is implemented as we expect).

To use the NetworkCredential class with both Basic and NTLM authentication it's pretty easy... Let's first update the code in our Form Click event so it sets the correct attributes on our RestClient object, (new code is in blue):

private void cmdGO_Click(object sender, EventArgs e)
        {
            RestClient rClient = new RestClient();
            rClient.endPoint = txtRequestURI.Text;
            
            if(rdoBasicAuth.Checked)
            {
                rClient.authType = authenticationType.Basic;
                debugOutput("authenticationType.Basic");
            }
            else
            {
                rClient.authType = authenticationType.NTLM;
                debugOutput("authenticationType.NTLM");
            }

            if(rdoRollOwn.Checked)
            {
                rClient.authTech = autheticationTechnique.RollYourOwn;
                debugOutput("autheticationTechnique.RollYourOwn;");
            }
            else
            {
                rClient.authTech = autheticationTechnique.NetworkCredential;
                debugOutput("autheticationTechnique.NetworkCredential");
            }


            rClient.userName = txtUserName.Text;
            rClient.userPassword = txtPassword.Text;

            debugOutput("REst Client Created");

            string strResponse = string.Empty;

            strResponse = rClient.makeRequest();

            debugOutput(strResponse);

Next we just add a simple if clause to our RestClient class:

  • If we want to use the "Self-Rolled" technique, then we default to basic authentication
  • Else if we want to use NetworkCredential Class then we let it take care of the Authentication Type (Basic or NTLM) - that's the power of using it!

Note when we come on to the CredentialCache we will use the attribute "authType".


using System;
using System.IO;
using System.Net;


namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    public enum authenticationType
    {
        Basic,
        NTLM
    }

    public enum autheticationTechnique
    {
        RollYourOwn,
        NetworkCredential
    }

    class RestClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }
        public authenticationType authType { get; set; }
        public autheticationTechnique authTech { get; set; }
        public string userName { get; set; }
        public string userPassword { get; set; }


        public RestClient()
        {
            endPoint = string.Empty;
            httpMethod = httpVerb.GET;
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();

            if (authTech == autheticationTechnique.RollYourOwn)
            {
                //We'll only do Basic Authentication if we roll our own
                String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
                request.Headers.Add("Authorization", "Basic " + authHeaer);
            }
            else
            {
                NetworkCredential netCred = new NetworkCredential(userName, userPassword);
                request.Credentials = netCred;
            }
            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
        }

    }

The only real code of interest is the NetworkCredential class its self:

  • We simply create a new instance of one, passing the username and password in the constructor
  • We then set the "credentials" attribute of our request object to our newly created NetworkCredential object

Lets' test with our client!

Test 1: API Set to Basic - Client Using Self-Rolled Header

vp-6 test1.png

Result: Pass!

Test 2: API Set to Basic - Client Using NetworkCredential Class

vp-6 test2

Result: Pass!

Test 3: API Set to Windows (NTLM) - Client Using Self Rolled Header

vp-6 test3

Result: Fail! - Technically this is not a fail as the software is behaving exactly as we expect, but just laboring the point that our self-rolled header does not allow us to authenticate when the api is using Windows authentication.

Test 4: API Set to Windows (NTLM) - Client Using NetworkCredential Class

vp-6 test4.png

Result: Pass! We have for the first time, authenticated to a Windows, (NTLM), protected API using the NetworkCredential class.

You can see the power and simplicity of this approach, but let's try one more test...

Test 5: JIRA API Set to "Basic" - Client Using NetworkCredential Class

As mentioned previously when using our Jira API - the self rolled approach worked fine, let's try with the NetworkCredential Class:

vp-6 test5

Result: Fail! - I actually would call this a fail this time. I think this should work. As mentioned above though, Atlassian did not implement their API according to the true nature of Basic Authentication.

Test 6: JIRA API Set to "Basic" - Client Using Self Rolled Header

vp-6 test6

Result: Pass! We knew this already though!

CredentialCache

I was going to write about the CredentialCache but the MSDN entry on it says everything I was going to say anyway! It can be found HERE.

In short:

  • Used to store multiple credentials
  • You assign it to HttpWebRequest Credentials attribute the same way we assigned the NetWorkCredential
  • When Adding multiple credentials you can assign the "Type" of authenticaton, e.g. Basic, NTLM etc, see below:

vp-6 Credential Cache

References

The unique reference for this tutorial is: VP-6

The Video

Consuming a REST API from c#

User Story

As a developer
I want to call a RESTful API from my c# application
So that I can consume and use the resources it provides

Overview

I’m guessing if you’re reading this that you probably have some idea what a RESTful API is, so I won’t delve too deeply into the why’s and wherefores, however some key points to note:

  • Centers around the concept of “resources” that you consume, create, update and destroy
  • Accessed via a “base” URI
  • Can return data in any format, (XML, atom etc.), but most usually JSON
  • Usually accessed over HTTP with the usual string of verbs: GET, POST PUT etc.
  • Platform agnostic
  • Lightweight (and usually fast!)

Ingredients

For this tutorial you will need the following, (or something similar):

  • Windows PC (I’m running Windows 10 on a Surface Pro 3)
  • Visual Studio Community Edition
  • Internet Access (if you want to use the sample REST service – if you’re reading this then I guess this is already sorted!)
  • 30-40 minutes of your time

Ok – Let’s get started!

Test the REST

Before we launch into writing code to access your REST service, you should simply be able to access it directly from your browser and observe the data returned, (in essence that’s really all this tutorial is – making a http request from c# code!).

Now as I’m a nice guy I’ve placed an example, (and very simple!), RESTful service on the interweb for you to play with, which coincidentally is the REST API I’m using for this tutorial.

Note: The API is read only, you’ll only be able to list all items in the db, or retrieve a particular item, (adding, updating and removing actions have all be disabled).

The API Can be found here:

http://dry-cliffs-19849.herokuapp.com/

This should take you to the “home-page” of the REST API, note that this is not really anything to do with the REST API its self, it’s just a nice landing page. (I say nice, the “styling” as you can see is straight out of the early 90s).

You will see a listing of all the REST Actions (and associated HTTP verbs), that will allow you to use our User resource.

HTTP Request URI Action Purpose
GET http://<base uri>/users index List all User resources
GET http://<base uri>/users/n show Get the user resource with an id of n
POST http://<base uri>/users create Create a new User resource
PATCH http://<base uri>/users/n update Update User resource with an id of n
DELETE http://<base uri>/users/n destroy Delete User resource with an id of n

The first 2 actions are activated and by clicking on the associated links you will be presented with the JSON response. For the first request, (listing all User resources), you should see something similar in your browser:

SNAGHTML250897

What you are seeing is a list of JSON objects representing a list of users in the database. Again not wanting to “cop out”, but I won’t go into too much detail about JSON, (I’m assuming if you’re reading this you have a basic understanding at least), however it’s really not that difficult…

Using an excellent tool like “jsoneditoronline.org”, you can cut and paste the JSON our test RESTService is returning to get a better idea of its anatomy, this will be useful for later:

SNAGHTML291e3e

As you see we were returned an array, (the ‘[‘ opening bracket denotes this), of user objects. Each object starts with a ‘{‘ and ends with a ‘}’.  Within the object we then have a list of key / value pairs, e.g. :

Key Example Value
id 3
name Adam Smith
email adam@email.com
nationality Scottish

It’s fairly easy now to understand the structure, and while it can get somewhat more complex in other instances, (e.g. when the value is another array of key value pairs), it’s essentially just the same design pattern. For the purposes of this tutorial it should suffice.

With all that done, (i.e. you’ve “tested” that you can access the REST service via your browser), it’s now time to start designing our solution.

User Interface Design

This section title is probably a little grandiose for what we are doing here, but I like to follow this pattern for even the simplest apps – that being mocking up the simple interface. I tend to think more visually so it’s only when I do this that I actually start to think how things will work…

The Mock Ups

I’ve used an excellent tool called “Balsamiq” to create the following mocks that depict the client before and after  making a call to the rest service. It is this interface we’ll want to construct in Visual Studio.

mocks

The eventuation of this amazing design will look something like:

actualclient

Sequence Diagram

Using some fairly rudimentary UML, the sequencing for the application looks as follows:

blog-11 sequence diagram

Probably the only thing of note is that our RESTClient, (i.e. our c# app), makes a synchronous call to our REST API… I.e. it waits for the response payload. This is probably not how you’d want to architect this in the real world, (you’d make a asynchronous request to allow for any network latency etc..). For the purposes of the tutorial it will suffice, and making an asynchronous request can be a bit of extra credit homework!

Again this is a very simple request response model, and possibly doesn’t require the need for a UML sequence diagram, however I think it’s just good practice, (especially when you are eventually faced with more complex sequencing!).

RESTClient Class Design

Ok now we’re getting down to it!

For the purposes of this tutorial we’re going to write a custom c# class that encapsulates functionality from multiple standard .net libraries. The class we’ll build is as follows:

Blog-11 RESTCLient Class

First we use an “enumeration” to map to our list of possible HTTP Verbs, you’ll see here we’ve only mapped GET & POST, and in fact we’ll actually only be using GET in this tutorial.

The actual class its self is pretty simple, containing 2 attributes:

  • endPoint (this is the URI we’re making the request to)
  • httpMethod (as describe above this just represents our HTTP Verb)

The class then has a very simple constructor and 1 additional method:

  • makeRequest (yes you’ve guessed it this initiates the request to our REST Api!).

Coding

Ok we’re finally here! It’s time to code.

Start Visual studio and create a new c# Windows Application – call it whatever you like.

Ok now we have a nice empty Windows form, you’ll want to place the following UI elements onto it:

  • TextBox to hold the Request URI (I called mine: txtRequestURI)
  • TextBox to hold the Response Payload (I called mine: txtResponse)
    • Set Multiline Property to true
    • Set ScrollBars Property to vertical
  • Button (I called mine: cmdGo)
  • Label
    • Set name Property to Request URI:
  • Label
    • Set name Property to Response:

With the form design complete, we can now move onto the creation of our RESTClient Class.

In Visual Studio right click your project, then select: Add -> Class..

snaghtml9c71c

When the Add New Item window appears, select Class (1), give it a name (2) – I called mine RESTClient.cs then click add (3):

untitled-picture

Your solution hierarchy should look somewhat like the following:

2016-09-30_12-37-24

Now add the following code to your RESTClient class, (or whatever you called it):

UPDATED: Used a Try / Catch / Finally Block instead to Using so we capture Non-successful responses.

using System;
using System.IO;            //Needs to be added
using System.Net;           //Needs to be added
using System.Text;          //Needs to be added

namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    class RESTClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }   

        //Default Constructor
        public RESTClient()
        {
            endPoint = "";
            httpMethod = httpVerb.GET;
            
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            var request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();
            
            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                //We catch non Http 200 responses here.
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
            
        }
    }
}
  •  The constructor basically sets our class attributes, (and as mentioned above we are defaulting to using the GET verb).
  • The makeRequest method creates a "request" object which is basically derived from the standard c# HttpWebRequest object, we then:
    • Make a Request
    • Wait for the response, (synchronous)
    • Check to see if there was an error
    • If no error then we parse the response and return it back to the caller as a string

Debug Output

Visual Studio has a range of tools to help you debug your code, (and they're excellent), but most of the time I like to include my own little function that writes out the contents of variables, or even just where I am in the code. I also use this function to write out things like our return string for the RESTClient class, feel free to use it it's pretty self explanatory:

  • It takes the string you want to write out, and send it to:
    • The Visual Studio Debug window
    • Any other text box you like, (in this example it's called txtResponse)
  • In order that the output "scrolls" to the end of the text box, you need to "ScrollToCaret"
private void debugOutput(string strDebugText)
{
  try
  {
    System.Diagnostics.Debug.Write(strDebugText + Environment.NewLine);
    txtResponse.Text = txtResponse.Text + strDebugText + Environment.NewLine;
    txtResponse.SelectionStart = txtResponse.TextLength;
    txtResponse.ScrollToCaret();
  }
  catch (Exception ex)
  {
    System.Diagnostics.Debug.Write(ex.Message, ToString() + Environment.NewLine);
  }
}

Wire Up the Button Click Event Handler

Finally we just need to create an instance of our RESTClient and make a call to makeRequest. To do this, simply go to the form designer and double click on the "GO!" button, this will create the default click event handler, then place the following code inside:

 private void cmdGo_Click(object sender, EventArgs e)
{
  RESTClient rClient = new RESTClient();

  rClient.endPoint = txtRequestURI.Text;
  debugOutput("RESTClient Object created.");

  string strJSON = string.Empty;

  strJSON = rClient.makeRequest();

  debugOutput(strJSON);

}

All it does is:

  • Instantiate a new instance of our RESTCLient
  • Set the end point to whatever we want to call
  • Prepare our response string
  • Make the call to the end point using our makeRequest function
  • Write the output to our text box, (and the Visual Studio debug window)

Give It A Go!

Run your project (F5) and try both of the following Request URI's:

In both cases you should get a response back from the REST service with HTML and JSON respectively.

2016-09-30_13-06-06

That's It!

Next Time...

Next time we'll tack the JSON response and work with it in c#

Push a New Git Repository to Github

Go to github and select “New Repository”

new repo

Enter the name of your repository (this needs to be unique to your github profile)

2016-04-13_21-17-47

Click “Create Repository”

Screen Shot 2016-04-13 at 8.50.38 PM

TIP: When you create a repo in Github, they basically give you the step by step instructions required at the local command line to initiate a new local repo and push it up to github:

2016-04-13_21-18-17

Go to your PC/Mac and fire up a command line…

Navigate into the directory of your software project:

2016-04-26_20-36-05

Initialise you local git repo (if there isn’t one).

git init

This command initiates a new local git db that is used for managing your source code, you should see a successful initiation message as detailed below:

gitinit

If you type the following, you'll see the current status of your local git repository:

git status

2016-05-03_20-57-18\

This shows us that while the repository has been initiated, none of the files (and directories), are being "tracked". This means they are not under git version control. To rectify this, issue the following command to add all files (and directories), for git tracking:

git add .

Get the status of your repository now and you will see that all files are being tracked:

2016-05-03_21-02-29

We now need to "commit" the changes we've made, (in this instance it's just adding all files to git version control), by issuing the following command, (the -m is message flag that allows us to add comments for this commit, this helps us track which files have been affected by a particular commit).

git commit -m "This is our first commit"

2016-05-03_21-07-42

Follow this with another status report on our repository (git status), and you should be told that there is nothing more to commit, and the working directory is clean...

At this stage our local git repository is up to date, (and if you were just a lone developer using git for version control you'd be "done"). However our local git repository is still not "associated" with the remote repository we created on github, (git and github are different things so don't confuse them!).

To make that association between your local git repository and the remote repository, type the following at your command line, (your's will look slightly different depending on the name of the repository you created on github):

git add remote origin https://github.com/binarythistle/aggregator.git

TIP: Remember the kind info the folks at github gave you when you created your github remote repository (see above)? Well this just the 2nd to last line.

Finally, issue the "push" command to push your local git repository to your newly associated github repository:

git push -u origin master

You'll be asked to authenticate to github when you perform a remote push...

Once the push is finished you're done! Both repositories, (local git and remote github are in sync).

 

Add Bootstrap to a Rails Project

  1. Add the bootstrap-sass gem to the project GEMFILE
gem 'bootstrap-sass'

2. Bundle Install  to install the gem:

bundle install

3. Add the following input directives to any of the .css.scss files in the "stylesheets" folder of the asset pipeline. (I usually create a dedicated file for this purpose).

@import "bootstrap-sprockets";
@import "bootstrap";

4. To utilise the bootstrap javascript libraries, (you'll require this for things like drop down menus etc.), you'll need to include the line, (I just put in the main application.js file):

= require bootstrap

SNAGHTML8cab4a

You're Done!

Tip: Remember that scaffolding code will create a default set of css and js files that can conflict with bootstrap styling. If in doubt delete the bootstapped files.

Pull a New Repository from Github

1. Go to Github.com and find the repository you want.

2016-04-13_21-20-27

2. Select “https” as the retrieval method (it’s simpler!)

2016-04-21_20-55-12

3. Copy the down load link

You’re Done with Github!

  1. Go to your local PC / Mac and open a command line (git needs to be installed on your local machine)
  2. Type:
git clone <pasted https url copied from github.com>

You're done pulling down a copy of the repository!

List the contents of your current directory, and you'll see a folder for your new repository.

Change into this directory and you can issue the "git pull" command to do a refresh check on any subsequent updates on github.

Micro-Vid Here!