Introduction to Entity Framework

Prerequisites

You’re in luck this time, there aren’t really any prerequisites! (Some c# coding would of course be useful…)

Code is available on Github HERE

User Story

As a developer

I want to map my application objects seamlessly to my database

So that I don’t have to worry about manual CRUD* operations

* Create, Read, Update and Delete

 

What You’ll Learn

When done, you should be able to:

  • Understand what Entity Framework (EF) is
  • Understand what “code first” and “database” first approaches are
  • Perform CRUD operations on a database using c# and Entity Framework
  • Understand what Linq is, and how it relates to Entity Framework

 

Ingredients

  • Visual Studio (I use Community Edition) – Free
  • SQL Server (I use Express) – Free
  • ~ 20 Minutes of your time

 

My Lab set Up

No need this time as the set up is super simple…

easy office space GIF-downsized_large

 

Our Application

The little application we’re going to write from scratch is called ” Task Master”, don’t get too excited though…

sad face GIF-downsized_large

With it we can:

  • Create simple tasks that get added to our DB
  • Read in our Task List and display, (bind), to a data grid
  • Update (Edit) an existing Task
  • Delete an existing Task

Each Task will have the following attributes:

  • Description
  • Due Date
  • Status (this will be a simple list of values from our DB too)
    • To Do
    • In Progress
    • Done

 

Wireframe

The user interface of our Windows forms app is “functional” at best… It literally is there only to test our main CRUD functions, so I’m not suggesting it’s something you’d adopt in your actual real world apps!

Idle State

Clean Form

  • What our form will look like after start up
  • If we have any existing tasks they will be loaded into our data grid (we’ll using “binding” here)

Mid-Edit Stage

Edit Form

  • We have clicked on an existing task and clicked “Update” (note Update button now reads “Save”)
  • Values are populated into our form elements
  • We can either click:
    • Cancel – to cancel the edit
    • Save – any changes we made to the task detail is saved back
    • Delete – self explanatory
  • In all cases we revert the “Save” button back to “Update” and clear the form fields

IMPORTANT: We’re not going to expend too much time “augmenting” the UI to ensure incorrect button presses cannot occur, (enable / disable buttons etc.) it adds too much “fluff” to the project…

 

Class Diagram

The primary classes that will form the basis of this project are shown below:

class 2

tmDBContext

If you take anything away from this tutorial it would be the “DbContext” class, (you’ll see that our “tmDBContext” class inherits from this).

From MSDN:

The Entity Framework maps the entities and relationships that are defined in your model to a database. The Entity Framework provides facilities to do the following: materialize data returned from the database as entity objects; track changes that were made to the objects; handle concurrency; propagate object changes back to the database; and bind objects to controls.

The primary class that is responsible for interacting with data as objects is System.Data.Entity.DbContext (often referred to as context). The context class manages the entity objects during run time, which includes populating objects with data from a database, change tracking, and persisting data to the database.

Basically, it’s a big deal – more on this when we code!

Status

The Status class basically represents the different statuses that out task can have, you’ll notice that the tmDBContext class has both Status and Task as properties… You’ll notice that we’ve over ridden the “ToString()” method of this class there’s a good reason for this as you’ll find out when we start to code.

Task

The Task class models the, (surprise, surprise!), tasks that can be added to our db. You’ll notice that it has both:

  • StatusID
  • Status (class)

again more on this later.

 

 

Database Schema

Our very complex database schema is shown below:

schema

Look kind of familiar?

Well it should! It matches almost exactly the Task and Status classes shown above. So where does Entity Framework fit into all this? Well a picture paints a thousand words:

2018-04-17_19-57-18

It basically sits in between the code-based classes and the DB… But what exactly is Entity Framework? I’m guessing if you’re here you kind of have an idea, but here’s a few bullet points of clarification, (to be honest the real clarification will come with the code!):

  • It’s a “persistence” framework
  • It’s a layer of abstraction between the application, (code), and the database
  • You may also hear it referred to as an “ORM” – Object Relational Mapper
    • It is, but really Persistence Frameworks use ORM’s

Awesome I hear you cry,  but why would I use EF, or more especially what are the advantages to using EF? There’s a great article HERE, so have a read, but to summarize:

  • Productivity
  • Application Design
  • Code Reuse
  • Application Maintainability
Which Came First: The Chicken or the Egg?

catch self GIF-downsized_large

Or more correctly, which came first: the code or the DB?

For some of you this may seem obvious, to which you’d answer: “the database!” This of course is not incorrect, (apologies for the double-negative), but it’s not a clear cut as that…

With Entity Framework we basically have 2 broad approaches on how we could proceed:

  1. Database First (see answer above)
  2. Code First (this tutorial will use this approach)
Database First

With database first, we start with the database and “generate” our code classes from this DB schema. (EF provides a “wizard” that gives us this functionality).

db first

This is a totally valid approach, and you may be “forced” to use it in instances where your organization has DBAs and they “control” the DB and schema. You may also opt to use this approach when you have an existing DB that you need to build an application around, (not really ideal for a whole number of reasons, but hey, we live in the real world and this may just be the reality of it).

We are not going to cover this approach here.

Code First

With code first we design and write our classes in the code first, (shock!), and then using something called “migrations” we gradually create our db schema from that.

code first

This is a far more “exciting” approach, but may seem quite alien to some of you. For me the primary benefits of this approach are:

  • Better application design (assuming you think about your classes!)
  • Incremental DB changes, (via the multiple migrations you create), that can then be replicated across environments: Dev, Test, Prod etc

DBAs will of course be horrified by this approach, but this article is not about them! You may also here that this approach is less flexible and you don’t have the same configuration options over the db… So far I’ve not really found that to be true, but I’m sure there are some obscure examples that someone may want to share…

Anyhow this is the approach we are going use, so get ready!

Alternatives

Before we get into the coding, for some reason I always like to be aware of any “alternatives” – not sure why, I think it just helps me contextualize things…

Wikipedia lists a number of frameworks here, (you’ll notice they refer to them as ORMs).

Personally, I’ve only used 1 of the others listed there, and heard a whole lot about another…

Linq to SQL

Interestingly another Persistence Framework from Microsoft, and one that I used in a “DB First” application. Works fine and shares a lot of similarities with EF, be warned however that Microsoft’s strategic persistence framework is Entity Framework, with Linq to SQL being gradually side-lined. Time to migrate!

nHibernate

Never used used this, but is a major contender in the space…

 

How does Linq Fit?

Finally, before we actually start coding I just wanted to say a little bit about “Linq”, (not “Linq to SQL” as mentioned above, just plain old “Linq”), and how that fits into the picture – primarily because when I first started I got a bit confused…

So what is Linq?

  • Stands for: Language Integrated Query
  • It’s not a Persistence Framework
  • It’s a ubiquitous way to query data from within your code
  • You can use the same query syntax to query different data sources

Put simply, I used Linq queries to query my data source when I was using “Linq To SQL” as my persistence framework, (the name gives it away!). But we also use Linq in a very similar, (in fact exactly the same), way with EF.

Linq allows us to write the same query “in code” that talks to different data sources, e.g.:

  • Realational DBs
  • XML

Or even the same Data Source, (e.g. MS SQL Server), but via different persistence frameworks, e.g.

  • Linq to SQL
  • Entity Framework

It abstracts us away from things like SQL, (relational DB’s), or XQuery, (XML), and allows us to use the same syntax.

Hope that was clear? If not moving to the coding should help clarify!

 

Now we dev!

OK lets begin with the easy bit and get our form user interface set up. As mentioned above it’s purely for testing the EF functionality we’re developing so feel free to come up with something else!

I take it for granted that you’ve created a new Windows Forms project – I’ve named mine “TaskMaster”

Our Form

form

Form Element Naming

(I’ve grouped the buttons together, hopefully you can follow the formatting I’ve used!)

form elements

Setting Up Entity Framework

Install Entity Framework

OK we want to ensure Entity Framework is available to our project, so in Visual Studio:

  • Tools -> NuGet Package Manager -> Package Manger Console
  • At the Package Manager prompt (PM>) type:
PM> install-package entityframework

Hit <Enter> and the latest version of Entity Framework will be made available to your application:

pm ef install

I'd personally also just check the project references in solution explorer:

sol exporer

You should see:

  • EntityFramework
  • EntityFramework.SqlServer
  • System.ComponentModel.DataAnnotations

have been added.

Create Your DBContext Class

1st let's just get organised:

  • Right Click you Project
  • Select Add
  • New Folder
  • Name the Folder "Model"

Now:

  • Right Click the "Model" folder
  • Select Add->Class...
  • Name it "tmDBContext" as per the class diagram above

 

tmdbcontext

Now in the code of your tmDBContext class ensure:

  • You add a "using" reference to: System.Data.Entity
  • Ensure the tmDBContext class "inherits" from DbContext

Your code should look similar to the following:

using System.Data.Entity;

namespace TaskMaster.Model
{
  class tmDBContext : DbContext
  {
  }
}

We're not finished with our db context yet... but first we need to:

Update Our App.config file

The app.config file is where, (surprise, surprise), our applications configuration is stored. Now while Entity Framework is clever, we still need to set up a connection string to our database, this is then used by our tmDBContext class to connect.

In this example we're using MS SQL Server running locally on our dev machine. We're also using "Integrated Security" so we don't need t supply user credentials explicitly, (our current windows login account will be used).

So that being said my connection string looks like:

<connectionStrings>
 <add name="tmDBContext" connectionString="data source=DESKTOP-H958OBO\SQLEXPRESS;initial catalog=TaskMaster;integrated security=True;" providerName="System.Data.SqlClient"/>
</connectionStrings>

 

I add this string to the app.config file withing the main <configuration></configuration> tags:

connect

There are a few important things to note here:

  • The "Name" of the connection string should equal that of your tmDBContext class (for the purposes of this tutorial anyway - you can experiment with other variations later...)
  • The "data source" attribute will obviously need to reflect your own local SQL Server instance
  • The "initial catalog" attribute is the actual database that will house our entity tables. At the moment this does not yet exist on out SQL Server
Enable Migrations

As mentioned very briefly above we use "migrations" to incrementally make changes to our Database based on the code classes we develop, but we have to enable them first. To do so, go back to NuGet Package Manger Console and type:

PM> Enable-Migrations

You should see:

migrations ena

A folder called: "Migrations" will also appear under you project in solution explorer.

Note: Our database still does not exist yet...

Would you believe that's the plumbing for Entity Framework established? Yes it is! And you're probably not that impressed yet, ok well hold on!

Creating Our 1st "Code First Class"

So this is where the magic starts to happen... As per our class diagram above, we want to model 2 things:

  • Our Task
  • A list of Task Statuses

We'll start with our "Status" class first.

On the "Model" folder in solution explorer, right click then:

  • Add-> Class...
  • Name your class "Status"
  • And give it 2 attributes: Id (Int) and Name (String)

Your code should look like the following:

namespace TaskMaster.Model
{
 class Status
 {
 public int Id { get; set; }
 public string Name { get; set; }
 }
}

But we're still not done.

We need to tell Entity Framework that we want this class to be "replicated" to our DB and come under Entity Framework "control" (probably a bad use of words, but hey..)

To do this we go back to our tmDBContext Class and add our "Status" class as a "DbSet", the code of our tmDBContext class should now look like:

using System.Data.Entity;

namespace TaskMaster.Model
{
 class tmDBContext : DbContext
 {
   public DbSet<Status> Statuses { get; set; }
 }
}

Before I explain fully what's going on let's proceed to the real money shot, and get this baby in the DB. To do this we create a "Migration", so go to NuGet Package Manager Console and type:

PM> add-migration AddStatus

I've used "AddStatus", (no white-space), as the name of this particular migration as that's the most recent change we've made, but this name could be anything really. It's best to keep it meaningful though.

Again, our DB, (and table), have still not been created. But what has been created is a "Migration" file that's been placed in our migration folder:

status mig

And the contents of the file its self:

status ig cont

This migration file is:

  • Prefixed with a time stamp
  • Contains an "Up" method (What needs to be created)
  • Contains a "Down" method (what needs to be rolled back should we want to roll back the effects of that migration)

In short these files keep track of all the incremental changes to our DBContext monitored classes since the last migration was created, so if you've created 5 new classes since the last migration, this would be reflected in the latest migration file you'd generate. In our case, we've just added 1 class and brought into the line of sight of our tmDBContext class.

Finally it's time to run the migration!

Again, at the Package Manager prompt, type:

PM> update-database

You should see something similar to the following output:

update database

If we check our SQL Server DB...

db

So what just happened?

Step by step this is what we did, (you can re-use the "recipe" going forward):

  1. We installed Entity Framework
  2. We created our own "tmDBContext" Class that inherits from "DBContext"
  3. We updated the app.config file to include a connection string to our database server
  4. We "enabled migrations"
  5. We created a code-based class with 2 attributes
  6. We then "added" the class as a property of our tmDBContext Class, (go to the class diagram above to see this relationship). We used "DBSet" to achieve this.
  7. We added a migration, this creates a list of commands to be run against the db in order to create a db entity matching any "modeled" classes in our tmDBContext class
  8. We then update the database

Steps 1-4 you only have to do once per project, (unless you're connecting to multiple databases...)

Steps 5-8 will be performed multiple times throughout the development of the project, (indeed smaller change increments are a good thing!).

Seeding Data

The work flow of a Task in our system is very simple:

status

So we want our user to be able to select one of these statuses, (from our dropdown) when they create or update a task they've added to the db.

We could:

  • Hard code these values in our app
  • Go to our DB and add values to our Status table (using SQL or Management Studio)
  • Create a migration and add the values form there

If any of you picked the 1st option, shame on you! This is bad practice for so many reasons I won't go into here

2nd option is better, but what about when we move from our Development db to our Test DB or even "production-replica" db? We'd have to add those values again manually...

We'll of course use the 3rd option and use a Entity Framework to create a migration that will add the values for us. This means when we want to stand up another instance of our DB, we just run all the migrations and the table values will be added also.

Create an "Empty Migration"

Go to NuGet Package manager console and type:

PM>add-migration seedStatusData

Again the name of the migration could be anything, but best to make it meaningful...

This will create another migration file, with empty Up and Down Methods, open this file and add to the Up method the following code:

Sql("INSERT INTO Status (Name) VALUES ('To Do');");
Sql("INSERT INTO Status (Name) VALUES ('In Progress');");
Sql("INSERT INTO Status (Name) VALUES ('Done');");

Your migration file should look like this:

seedMigration

Now I won't over explain this, but basically we've effectively put sql into our migration file, so when it's run can you guess what happens?

Well, lets see... Make sure you save the file then type the following at NUGet Package Manger Console, (NPMC):

PM> update-database

You should get a success message and if we directly query the "Status" table in our DB we should see:

select from db

Note: I've put square brackets round the table name as "Status" is a reserved work in TSQL. (In retrospect this was probably a bad name for our class/table).

You'll also note that as the ID column had "identity" set to true as well as being a primary key, (see 1st migration above), we did not explicitly have to provide values for the ID in our 2nd "seed" migration as the DB creates these values for us.

Migrating our Task Class

OK it should now be relatively straightforward to add, (in code), our "Task" class and migrate to our DB - and indeed it is!

On the "Model" folder in solution explorer, right click then:

  • Add-> Class...
  • Name your class "Task"
  • And give it 5 Attributes:
    • Id (Int)
    • Name (string)
    • Status (Status. I.e. the status class we defined above)
    • StatusId (int)
    • DueDate (DateTime)

so your code should look like this:

namespace TaskMaster.Model
{
  class Task
  {
    public int Id { get; set; }
    public string Name { get; set; }
    public Status Status { get; set; }
    public int StatusId { get; set; }
    public DateTime? DueDate { get; set; }
  }
}

The Id, Name and DueDate attributes are all very self explanatory, so I won't insult your intelligence by detailing what they are, (the only point of note is that we have made out "DueDate" null-able by declaring it with a "?"...

Navigation Properties and Foreign Keys

The other two properties that I do want to briefly discuss are:

  • StatusId (this is a Foreign Key attribute)
  • Status (This is a "Navigation Property")

These 2 properties are really a way for Entity Framework to maintain the relationship between the Task and Status entities, see our DB schema below:

schema

Now relationships, (or associations), and for that matter Navigation Properties are an expansive area that I don't want to go into too much more detail about here... If you are interested in more detail there's a great article HERE on the MSDN site, I suggest you read that.

In short though, these properties are a way for us to associate the Task to a given Status. It's also worth noting that the Navigation property, ("Status"), is not created in the database, it's used only at the Entity Framework level...

Important: Entity Framework uses something called "convention over configuration" which in short means that if you follow certain naming standards, e.g. "Id" for primary keys and "<classname>Id", e.g. StatusId for the foreign keys, it will scaffold and set up relationships between those classes (or entities if you prefer) as standard.

You can deviate from these standards but it will require you to write more code...

So with that let's migrate!

In NPMC type:

PM> add-migration addTask

Ooops! What's wrong with this picture?

2018-05-05_12-56-17

Yes that's right we forgot to bring the "Task" class we've just created into "view" of our DBContext.

Before we do that though, what can we do about our "blank" migration..?

Deleting, Overwriting and Rolling-Back Migrations

There are a number of scenarios where you may want to just delete the migration, (we'll do that in this instance), Overwrite an existing migration, (we could do that too), or actually roll back a migration that you've run.

Deleting

Before you just "delete" a migration file, (as simple as right clicking the file and selecting delete), you should first check to see if it's be run against the DB, (in which case you'd want to first roll back).

In NPMC, type:

PM> get-migrations

This gets the list of migrations that have been applied against the db, see below:

2018-05-05_13-04-53

We can see here that while we have 3 migration files, (including our new, but empty "addTask" migration), we have only run 2 against the DB. So in this case we can just delete that migration file and start again.

So let's do that, (right click and delete it!)

Overwriting

You could also just "overwrite" that empty migration with new one that is not blank, (once we add the Task class to our DBContext). If you wanted to take that approach you'd type the following in NPMC, (after DBContext code changes of course!):

PM> add-migration addTask -Force

Again note you should only do this if the migration has not been run against the DB!

Rolling Back

There may be instances where you have applied a migration, and realised that what you've applied is incorrect and you want to roll back. In this case you'd:

  • get the list of migrations you've applied, (possibly optional if you already know the name of the last migration you want to roll back to)
  • Update the database to your selected migration point, or "Target Migration"

So let's say we want to roll back the data we seeded to our db, you would do the following in NPMC:

2018-05-05_13-18-04

You can see that we "targeted" the migration we wanted to roll back to, and the framework then "reverted" all migrations that came after, in this case it was just our "seedStatusData" migration.

Note: if you've tried that out as an example, we DO actually want our Status data in data in there, so just make sure you update the database again to put it all back, hint: just "update-database"...

Update DBContext

Ok so the whole point of deliberately making a mistake was to show you how to play with migrations if yu do make a mistake. It should also re-enforce how important the DBContext class is, so with that let's update it:

using System.Data.Entity; 
namespace TaskMaster.Model 
{ 
  class tmDBContext : DbContext 
  { 
    public DbSet<Status> Statuses { get; set; } 
    public DbSet<Task> Tasks { get; set; }
  } 
}

Save the file then, we add a migration:

PM> add-migration addTask

Huzzah, the generated migration file now has "Stuff" in it:

2018-05-05_13-26-10

We then update our database and check that it was applied:

2018-05-05_13-29-09

If we actually go to SQL Server Enterprise manager and refresh our DB tables we should see our newly created table:

2018-05-05_13-30-50

Checkpoint: Ok we've covered a lot! But that's it for migrations etc. we'll now move onto our 2nd phase of coding and start working with Entity Framework classes to: Create, Read, Update and Delete Task data in our DB.

Reading Data From the DB (Pt1)

Ok so we've gone to all the bother of setting up a status table and populating that data, we now want to read that data into the items collection in our combobox.

The steps we have to follow to achieve this are:

  • Create and instance of our tmDBContext class
  • Get a "List" of our statuses (from the DB)
  • Populate the combo box

So the "code-behind" or form should look like:

using System;
using System.Data;
using System.Linq;
using System.Windows.Forms;
using TaskMaster.Model;

namespace TaskMaster
{
  public partial class Form1 : Form
  {
    private tmDBContext tmContext;

    public Form1()
    {
      InitializeComponent();

      tmContext = new tmDBContext();

      var statuses = tmContext.Statuses.ToList();

      foreach (Status s in statuses)
      {
        cboStatus.Items.Add(s);
      }
    }
  }
}

The only thing of note is that our "tmContext" gives us direct access to our entity classes (and more importantly that data "behind" them), without having to really having to do that much work - it's really simple (that's the power of the EF!)

We then just populate our combobox with each Task object in our list... But there's a problem, when we run our code and look in our drop down we get this:

Drop dpwn

While we have added our Status objects to the comboboxes items collection, when the list is rendered in the drop down, we're getting the object type name, not the Name property... To rectify this we'll "override" the ToString() method of the Task class so it defaults to returning the "Name" property whenever ToString() is called, the code should look like this.

namespace TaskMaster.Model
{
  class Status
  {
    public int Id { get; set; }
    public string Name { get; set; }

    public override string ToString()
    {
      return Name;
    }
  }
}

When we re-run our code, you should now have a list of the statuses we put in the DB.

Creating Data in Our DB

Ok, it's starting to get exciting now as we are going to create a task so that it will get added to our database!

Warning! We're going to start wiring up the events on our buttons, and while I'm going to put a little bit of "validation" in here, (e.g. not allowing us to create a task unless a status is selected), I'm not going to take this to any extreme as it's not the point of the tutorial.

Ok so on the form designer, double click our Create button, this should take you to the click event code...

2018-05-07_19-47-30

Into this event method, enter the following code:

if (cboStatus.SelectedItem != null && txtTask.Text != String.Empty)
{
  var newTask = new Model.Task
  {
    Name = txtTask.Text,
    StatusId = (cboStatus.SelectedItem as Model.Status).Id,
    DueDate = dateTimePicker1.Value
  };

tmContext.Tasks.Add(newTask);

tmContext.SaveChanges();

}
else
{
  MessageBox.Show("You must select a Task Status and enter a Task description.");
}

Let's test this now, so fire up our app, enter a description and select a task status and click the "Create" button.

If all is correct "nothing" should happen, (or nothing appears to happen at least), but if we check our database we get a different story...

2018-05-07_20-00-46

Note: We'll fix up our dodgy "UI experience" with the next bit of code...

So what did we do here?
  • We checked to see if we had a selected value for our Task status and that we had a non empty task description (the 1st "if" statement), otherwise we showed a message.
  • If the 1st if statement passed we then:
    • Created a "Task" object, (note I used "Model.Task" to declare this (for clarity)
    • Set the attributes of our new Task Object, note we had to cast the comboboxes selecteditem as a "Model.Status" in order to obtain the StatusId for our Task.
    • Called the tmContext.Task.Add() method in order to add our newly created task to the tmContext (it's important to realise that the Task has not been added to the DB at this stage, only to the the Entity Framework DBContext...
    • Finally we called the tmContext.SaveChanges() method. This actually writes all pending changes in our DBContext to the Db (we only have one).

Reading Data From the DB (Pt2)

Ok, so when we added a record to our db, (above), our app didn't give us any feedback that the record had been added, (even though it had!), which is a poor user experience.

In order to rectify this, we're going to "bind" our datagridview to our list of Tasks, (available via the tmDBContext obejct). Observe the diagram below to view 2 binding methods, (they are basically the same, 1 uses a "bindingcontext" object other does not:

binding source 2

While the 2nd option works, all the Microsoft documentation on binding to windows forms elements uses an intermediary "bindingsource" object, so that's what we'll use here.

If you want to read more on binding and in particular what the advantages of using a binding source are: I'd reccomend these 2 articles:

We're actually going to write a simple function that:

  • Creates a new bindingsource object
  • Using "Linq", we'll query our tmDBContect class for all Tasks and set the result to the bindingsource objects "DataSource" property, (see diagram above)
  • Set the DataGridView's "DataSource" property to the Binding Source object

Here's the function:

private void refreshGrid()
{
  BindingSource bi = new BindingSource();
 
  var query = from t in tmContext.Tasks
              orderby t.DueDate
              select new { t.Id, TaskName = t.Name, StatusName = t.Status.Name, t.DueDate };

  bi.DataSource = query.ToList();

  dataGridView1.DataSource = bi;
  dataGridView1.Refresh();
}

 

The only thing of any real novelty is the construction of our "query". For those of you familiar with SQL, while it's not the same, it has a similar feel to it, this however is Linq in action... (or more specifically Linq query syntax in action...)

We are effectively selecting every row from the tmContext class Tasks collection, and specifying the columns, (and column names), that we are going to use.

Not wanting to confuse things, but we could have used another Linq "syntax" to get the same result, this is called Linq Extension Method syntax, and it uses "lambda" expressions. I'm not going to cover that here but it is in the video associated to this post, or you can read about the differences here.

To see if this works we'll pack a call to the method in our form's constructor:

public Form1()
{
  InitializeComponent();

  tmContext = new tmDBContext();

  var statuses = tmContext.Statuses.ToList();

  foreach (Status s in statuses)
  {
    cboStatus.Items.Add(s);
  }

  refreshGrid();
}

We should also place a call to it in our CreateTask Button's click event handler after we call tmContext.SaveChanges() method, (this corrects our poor UI experience by updating the datagridview with any newly added Tasks)

All being well, when the form loads, we should see the Task we just added, in the previous code example:

binded data

Deleting Data

Ok we're now going to delete data from the db, to do this:

  • Quit the app if it's running
  • Double click the DeleteTask button to reveal it's event handler, and enter the following code:
var t = tmContext.Tasks.Find((int)dataGridView1.SelectedCells[0].Value);
tmContext.Tasks.Remove(t);
tmContext.SaveChanges();

refreshGrid();

You can probably figure out what's going on here as the code is quite self explanatory, but for completeness we're:

  • "Finding" the task we want to delete via our tmContext class
  • In order to do that, were obtaining the value of the "zero-index" cell of our selected row, (ensure you only have 1 row selected - you can enforce that via the DataGridView's properties)
  • We then remove the task from the tmContext (it's "tagged" for deletion although not yet removed from the DB)
  • As with creating Tasks, we call SaveChanges() to apply any pending changes in out fmContext class - this then applies the necessary changes to our db.
  • Finally we refresh the DataGridView

Updating Data

The actual act of updating data, is very simple and really just an extension of how we delete a Task... (except we don't delete the Task, we update it's elements), see code below:

var updateTask = tmContext.Tasks.Find((int)dataGridView1.SelectedCells[0].Value);

updateTask.Name = txtTask.Text;
updateTask.StatusId = (cboStatus.SelectedItem as Status).Id;
updateTask.DueDate = dateTimePicker1.Value;

tmContext.SaveChanges();

refreshGrid();

 

Essentially we:

  • Find the task we want to update
  • Update it's attributes
  • Save Changes to the DB
  • Refresh Our Grid

The user experience I've designed to enable updates in our app is not great but it does allow you to update items, the work-flow is as follows:

  • Select the row you want to update
  • Click the cmdUpdateTasks button
  • We populate our form elements with the relevant values
  • We change the text of cmdUpdateTasks button to "Save"
  • Click the the cmdUpdateTasks button (we check it's "Text" value is "Save"
  • We then find the the selected task
  • Update it's values to whatever we've set in the form items
  • Save Changes to DB
  • Refresh Our Grid
  • Reset all our form elements

We also have the "Cancel" button which just "resets" our form mid-update flow.

Warning: This flow is clunky at best and can probably trip up and throw errors if you don't follow it correctly... However it's only real purpose was to show you how you update data, and if done correctly it does that!

The complete code for both the UpdateTask and CancelTask button event handlers is shown below, (it's also available on github).

private void cmdCancel_Click(object sender, EventArgs e)
{
  cmdUpdateTask.Text = "Update";
  txtTask.Text = string.Empty;
  dateTimePicker1.Value = DateTime.Now;
  cboStatus.Text = "Please Select...";
}

private void cmdUpdateTask_Click(object sender, EventArgs e)
{
  if (cmdUpdateTask.Text == "Update")
  {
    txtTask.Text = dataGridView1.SelectedCells[1].Value.ToString();
    dateTimePicker1.Value = (DateTime)dataGridView1.SelectedCells[3].Value;
    foreach (Status s in cboStatus.Items)
    {
      if (s.Name == dataGridView1.SelectedCells[2].Value.ToString())
      {
        cboStatus.SelectedItem = s;
      }
    }
    cmdUpdateTask.Text = "Save";
  }
  else if (cmdUpdateTask.Text == "Save")
  {
    var updateTask = tmContext.Tasks.Find((int)dataGridView1.SelectedCells[0].Value);

    updateTask.Name = txtTask.Text;
    updateTask.StatusId = (cboStatus.SelectedItem as Status).Id;
    updateTask.DueDate = dateTimePicker1.Value;

    tmContext.SaveChanges();

    refreshGrid();

    cmdUpdateTask.Text = "Update";
    txtTask.Text = string.Empty;
    dateTimePicker1.Value = DateTime.Now;
    cboStatus.Text = "Please Select...";
   }
 }

 

Final Thought

Well that was a long post! Entity Framework is a huge area, but I hope this introductory post has enabled you to understand what it is and how to start using it

 

References

Code on Github Here: https://github.com/binarythistle/VP-17-Intro-to-Entity-Framework

 

 

Advertisements

POSTing to a REST API with c#

Prerequisites

I do prefer to take things all the way from start to finish, step by step, but sometimes, (in this instance!), it would mean repeating a lot of stuff that I’ve already covered… So if you’ve not already done so, I’d advise reading the following posts, (in order), to get yourself to this stage:

And although not strictly necessary, if may be worthwhile getting up to speed with JSON as we’ll be using a JSON data structure in this tutorial as part of the POST request:

These posts also have links to the accompanying YouTube videos if you prefer that medium, OK so now on with the tutorial!

Video

If you prefer a more visual medium we have an associated video tutorial here:

User Story

As a developer
I want to make a POST request to a REST API
So that I can add a new data object to our repository

Acceptance Criteria

On completing this tutorial you should:

  • Be able to use c# to make a POST request to a REST API
  • Cause a data object to be added to our application repository

Ingredients

I used the following ingredients to write this post, if you don’t have the exact same ingredients then feel free to substitute it for an equivalent item:

  • Windows PC (I’m using Windows 10 on a self-built desktop)
  • Visual Studio Community Edition 2017 (This is free and available from Microsoft)
  • Web Browser (I primary use Firefox with the “Live HTTP Headers” add-on)
  • Notepad++ (I know it sounds a bit weird using this as well as Visual Studio, but I do like to have a separate notepad running too)
  • A REST API that can accept POST requests, in this instance I’m running:
    • VMWare Workstation with a Centos 7 Guest OS
    • Jira Software – Basically a project management tool used a lot by software development teams, more HERE.

Just looking through my list with the exception of the PC Hardware and Windows OS, everything else here was free, (I paid $10 for a Jira License), which is pretty amazing when you think about it!

If you’d rather not set up a local version of JIRA on a VM etc, you can sign-up for a Cloud Account on the Atlassian web-site.

Or there are other “test REST API” web sites out there that you can dev / test against. I’ve not used any of these so can’t recommend one, if you can though then please leave a comment!

My Lab Set Up

As described in the ingredients section, here’s how my lab is set up:

VP-10 POST Request Lab Set up.png

User Interface Design

We are following directly on from the Authentication to a REST API tutorial, but I’ve made some cosmetic changes to the UI, the primary one’s are:

  • Remove the various options that allow you to select the type of authentication (in this tutorial we’re just going to use BASIC authentication)
  • Added a ComboBox to allow us to select the HTTP Verb, (more about that below)
  • Added a Text Box for the “POST Data” we’ll send with our request

The wireframe with some, (helpful?), mark up is shown below:

VP-10 POST REquest Wireframe.png

Class Diagram

There’s nothing really new apart from the new “postJSON” attribute!

class diagram

HTTP Verbs

There’s already a wealth of stuff written about HTTP Verbs, (or Request Methods if you prefer), most notably the specification RFC7231, so I’m really not going to add much to that…

I have for completeness though, (and to save you a mouse click!), listed the main methods you’re likely to encounter and use below, (I’m guessing if you’re here you probably have an understanding of this already):

VERB CRUD* Description
GET Read Used to read or return a resource, (JSON, XML etc.). GET requests don’t change data so are considered “safe”
POST Create Used to create a resource at the target. We’ll be using a POST request to send a JSON payload to Jira in the hope we can create a “ticket” in the system, (they are usually referred to as “issues” in Jira).
PUT  Update / Replace PUT updates a resource, but replacing the old one in its entirety, it requires more network bandwidth than PATCH, (see below)
PATCH  Update / Modify PATCH updates only parts of a required resource, e.g. just update the firstname value of a user object. Requires less bandwidth than PUT
DELETE  Delete Self explanatory, used to delete a resource – use with caution!

* Acronym denoting the common operations of:
Create, Read, Update & Delete

Query Strings

OK so I just want to clarify the anatomy of a URL and the Query String component in particular as I feel this is useful when we come on to talk about the differences between GETing and POSTing data…

The Wikipedia entry HERE is actually great and goes into lots of detail, but I just wanted to reinforce the 2 very broad components of a typical URL, as an example take the following made up URL:

https://blahblah.com/rest/api/2/issue/VP-1?expand=true&maxResults=1000

Basically the “query string” is everything from the question mark, (?), onward. It’s a non-hierarchical string that typically consists of attribute-value pairs separated, (usually), by an ampersand: &. Therefore in the above example, we have 2 sets of attribute-value pairs:

ATTRIBUTE VALUE
expand true
maxResults 1000

Note: The “attribute” and the “value” are separated, (again usually), but an equals sign: =.

So why am I saying usually a lot?

Well while the anatomy of the overall URL is well defined in RFC-1738 by Sir Tim Berners-Lee, the nitty-gritty, (not a technical term), of the query string is “non-standardised”. Meaning that there could be different interpretations of how it could be implemented, (different to the example given above).

Anyway this is all rather academic, the main point(s) I’m trying to make are:

  • Most query strings adhere to the example format given above
  • They, (query strings), are used to pass “data” to the server
  • GET Requests are a great example of where they are used:
    • The above example is GETing issue “VP-1” and the attribute value pairs passed as part of that request are telling the Jira server how to additionally process the request
  • They really have NOTHING to do with POSTing data!

That’s right I’ve just spent this time telling you about query strings, but finish with the point that they are not really that relevant to POST requests, (POST requests can still have query strings though, but the actual data we are POSTing is not placed here..)

Confused? Read on!

 

POST Data

Before we go on, here’s the man himself, (Sir Tim), pretending to do some work, (much like I do Monday-Friday, don’t tell my employer…)

sir tim.gif

OK, moving on…

So I hear you ask, if we’re going to POST data to a server and ask it to create a new resource, why couldn’t we use query strings as the method to do that?

It’s a good question, but the main reasons we would not use the query string as a method to pass data are:

  • Physical limitations of URL total length
  • Security (URL + Query String are stored in your browsers history)
  • The structure of the data we would want to pass

Let’s take each one in turn:

Physical Limitations

There is no theoretical limit to the query string, but there are physical limits as imposed by your choice of web server and web browser, (Google this if you’re interested in actual values), so if we’re wanting to pass “a lot” of data these limitations would scupper our attempts. You may even get a 414 Request-URI Too Long HTTP Status Code! Eurgh!

414 Error

Security

Have you ever been planning a holiday with your girlfriend and been typing stuff into a web browser when it drops-down and shows previous URL destinations, including embarrassing search terms contained in the query string?

giphy-downsized.gif

No neither have I, (cough), but you get the point. While query strings may be encrypted over the wire if you’re using HTTPS etc, that query string can still be cached and retrieved from your browser history. Not advisable if you need to POST “sensitive” data…

Data Structure

So aside from size limits and “security concerns”, the last reason you may not want to use a query string is the potentially, complex, hierarchical structure of your data.

As shown in the query string example, the data is structured in a non-hierarchical way, so it doesn’t really lend it’s self to more complex data structures with nested attributes etc. It’s not to say that it would be impossible to construct some kind of schema, but just because you could do it, doesn’t mean to say it should be done. To paraphrase Chris Rock:

“You could drive a car with your knees, but it doesn’t make it a good f@#$ing idea!”

Where does our POST data go?

An even better question, in short our POST data is actually sent in the body of our request message. I think this is where we need to start to code…

 

Now We Code!

IMPORTANT: Remember, we’re starting from a “base” project here, (you need to follow Authenticating to a REST API to get to the starting point here).

Before we launch into coding proper let’s just set up our UI, by following the design of our wire frame:Actual Form.png

So with the exception of the red dots, (used so you can reference the item properties below), our web form is pretty simple, and does indeed follow our basic design.

If you’re wanting to follow along with the code exactly then refer to the UI element property settings below. Once done, were pretty much finished with the UI.

Form UI Properties

Simple Button Logic

We’ve added a couple of buttons to help when demoing the solution, (just saves a few mouse clicks but they add up!):

  • Copy
  • Clear

Double-click both buttons on the designer to wire up the event handlers for both, assuming you’ve named everything as I have above then code for each should look as follows:

 

 


private void cmdClear_Click(object sender, EventArgs e)
{
  txtResponse.Text = string.Empty;
}

private void cmdCopy_Click(object sender, EventArgs e)
{
  System.Windows.Forms.Clipboard.SetText(txtResponse.Text);
}

Nothing terribly magical there:

  • For the "Clear" button we just set the contents of out Response Text Box to "empty"
  • For the "Copy" button we "Set the Text" of the clipboard to the contents of the Response Text Box (makes it easier to paste any responses into our JSON editor)

What You've Been Waiting For!

So now we turn our attention to our RESTClient Class, the full definition is below, you'll notice I've taken out some redundant code from the previous tutorial, (essentially the stuff that decides with authentication mechanism to use). I've coloured any new code in BLUE, (apologies to those of you that are colour blind...)


using System;
using System.IO;
using System.Net;

namespace VP_10_Prototype
{
  public enum httpVerb
  {
    GET,
    POST,
    PUT,
    DELETE
  }

  public enum authenticationType
  {
    Basic,
    NTLM
  }


  class RestClient
  {
    public string endPoint { get; set; }
    public httpVerb httpMethod { get; set; }
    public authenticationType authType { get; set; }
    public string userName { get; set; }
    public string userPassword { get; set; }
    public string postJSON { get; set; } //New Attribute

    public RestClient()
    {
      endPoint = string.Empty;
    }

    public string makeRequest()
    {
      string strResponseValue = string.Empty;
      HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
      request.Method = httpMethod.ToString();
      
      String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
      request.Headers.Add("Authorization", "Basic " + authHeaer);
      

      //********* NEW CODE TO SUPPORT POSTING *********
      if (request.Method == "POST" && postJSON != string.Empty)
      {
        request.ContentType = "application/json"; //Really Important
        using (StreamWriter swJSONPayload = new StreamWriter(request.GetRequestStream()))
        {
          swJSONPayload.Write(postJSON);
          swJSONPayload.Close();
        }
      }

      HttpWebResponse response = null;

      try
      {
        response = (HttpWebResponse)request.GetResponse();
        //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._
        using (Stream responseStream = response.GetResponseStream())
        {
          if (responseStream != null)
          {
            using (StreamReader reader = new StreamReader(responseStream))
            {
              strResponseValue = reader.ReadToEnd();
            }
          }
        }
      }
      catch (Exception ex)
      {
        strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
      }
      finally
      {
        if (response != null)
        {
          ((IDisposable)response).Dispose();
        }
      }
    }//End of makeRequest
  }//End of Class
}

 

In reality the new code if very straight forward:

if (request.Method == "POST" && postJSON != string.Empty)  
{  
  request.ContentType = "application/json"; //Really Important  
  using (StreamWriter swJSONPayload = new StreamWriter(request.GetRequestStream()))  
  {  
    swJSONPayload.Write(postJSON);   
    swJSONPayload.Close();  
  }  
}

 

We:

  • Check to see that our HTTP Request Method is set to POST and that we have a non-empty string for our "payload" - you'll note there is no real error checking here...
  • We set the request "ContentType" to JSON. As I've noted in the comment this is important! If you don't provide this specifier for your format, (JSON, XML etc.), you may get a 415 "unsupported format" error - (I did when I didn't put this in)
  • We then call the "GetRequestStream" method on our HttpWebRequest object, here's what the MSDN article HERE has to say about it:

 

GetRequestStream

This is basically the whole crux of the matter, we open a stream so that we can "attach" & send our JSON data with our request object, (as discussed briefly above).

We then:

  • Write our JSON string to the stream
  • We close our stream
    • Because we create our SteamWriter object in a "Using" statement, the StreamWriter object should be cleanly Disposed (and hence closed). I just put it in because it makes me feel better
  • The rest of the code you should be familiar with, but we basically call GetResponse on our request object to actually make the request

Making The Call

Ok all that's left to do is to make the call, this is what the code of the GO button's click event looks like:

private void cmdGO_Click(object sender, EventArgs e)
{
  RestClient rClient = new RestClient();
  rClient.endPoint = txtRequestURI.Text;
  switch(cboVerb.Text)
  {
    case "POST":
      rClient.httpMethod = httpVerb.POST;
      rClient.postJSON = txtPOSTData.Text;
      break;
    default:
      rClient.httpMethod = httpVerb.GET;
      break;
  }
  rClient.userName = txtUserName.Text;
  rClient.userPassword = txtPassword.Text;
  string strResponse = string.Empty;
  strResponse = rClient.makeRequest();
  debugOutput(strResponse);
}

Again, the only new code is in BLUE, it just:

  • Checks to see what we've set the verb to in our Combo Box
  • If it's POST, then:
    • We set the RestClient objects httpMethod attribute
    • We set the RestClient objects postJSON attribute to the contents of our txtPOSTData textbox
  • Else we just default the httpMethod to GET
  • The rest of the code is as before, we make a makeRequest call

Testing

Ok so this last bit will come down to you! You'll have to understand the format of both:

  • The URL of the POST API you're using
  • The Data payload you want to send with it

You will have to consult with the API documentation for your particular API, as I'm using Jira and want to create an "issue", I'll use the following:

API URL: http://localhost:8080/rest/api/2/issue/

JSON POST Data:

{
 "fields": {
 "project": {
 "key": "TEST"
 },
 "summary": "No REST for the Wicked.",
 "issuetype": {
 "id": "10002"
 },
 "reporter": {
 "name": "binarythistle"
 }
 }
}

I'm not going into too much detail about the JSON payload, (I had to derive this from the Atlassian documentation), but it's fairly self explanatory. My next tutorial will actually be a deep dive on a number of the most useful JIRA API calls, in that I'll detail how you can determine the format of your POST data payloads.

What Happens Next?

Well after clicking "GO", (and assuming you've filled in everything correctly), you should get a success response. In the case of Jira, we're returned a JSON string that gives us some useful information about the newly created issue!

Success

Video

 

Deserializing JSON with c#

Foreword

This tutorial details how to deserialize* a JSON “string” into a c# object using JSON.net by Newtonsoft. It’s a stand alone tutorial, but follows logically from Consuming a REST API from c#.

* It pains me to use a ‘z’ here but I need to keep the US audiences and search engines happy!

If you prefer video to the written word, you can find the accompanying YouTube video here:

 

User Story

As a developer
I want to parse, (deserialize), a JSON string
So that I can work with c# objects

Acceptance Criteria

By the end of this tutorial you should be able to:

  • Understand what JSON is and how it’s structured
  • Deserialize JSON Strings

Ingredients

  • Windows PC (to run Visual Studio)
  • Visual Studio (I use the Community Edition)
  • JSON Editor Online @ http://jsoneditoronline.org/ (you can use an alternative I just like this one)
  • 20 Minutes of your time

User Interface Design

Using Balsamiq the following wire frame emerged and helped clarify my thinking, it’s pretty simple really… It takes and input, input is processed by clicking the button and the output is shown in the bottom plane. There’s a 2nd button that clears the output window.

A good wire frame should speak for its self though, so here you go:

VP-1 Mock Up

Wire Frame – Realized

So here’s what the wire frame translates to in the real world, I’ve place the form element names there if you’re wanting to use the same naming convention as me, (not sure why you would my naming convention is pretty rubbish!):

2017-07-19_21-16-08

Point to note, both textboxes are set to “Multiline: True” and with vertical scroll bars…

“Wire Up” The Buttons

Ok, so let’s do some simple stuff first, double click each button on the form designer to invoke their click event handlers.

Into the “cmdClear” button’s handler place the very simple code:

private void cmdClear_Click(object sender, EventArgs e)
{
  txtDebugOutput.Text = string.Empty;
}

If you don't understand what that's doing then you may struggle with the rest of the tutorial, so moving on!

I also, (as in previous tutorials), like to add my little "debugOutput" function that takes a string, displays it on a multi-line text box and "scrolls" to the next line, so further text can be displayed. So inside your Form code, add the following function, (it's not mandatory for the actual parsing of JSON, but it just looks nice):

private void debugOutput(string strDebugText)
{
  try
  {
    System.Diagnostics.Debug.Write(strDebugText + Environment.NewLine);
    txtDebugOutput.Text = txtDebugOutput.Text + strDebugText + Environment.NewLine;
    txtDebugOutput.SelectionStart = txtDebugOutput.TextLength;
    txtDebugOutput.ScrollToCaret();
  }
  catch(Exception ex)
  {
    System.Diagnostics.Debug.Write(ex.Message.ToString() + Environment.NewLine);
  }
}

Let's test the function by adding the following code to the cmdDeserialise Click event handler. Test it by typing something in txtInput, and clicking cmdDeserialise...

private void cmdDeserialise_Click(object sender, EventArgs e)
{
  debugOutput(txtInput.Text);
}

Yeah - amazing right?

giphy-downsized.gif

So what is JSON?

I'm going to guess if you're reading this you have some idea... So I don't really want to go over in vast detail what it is here as there are plenty of great written resources already, (check out the Wikipedia entry here).

What I do want to do though is labor some points that will come in useful when we start deserializing the raw JSON strings...

  • Stands for: "JavaScript Object Notation"
  • Open format used for the transmission of "object" data, (primarily), over the web.
  • It consists of attribute-value pairs, (see examples below)
  • A JSON Object can contain other "nested" objects

Anatomy of a Simple JSON Object

simple json

In the above example we have a "Person" object with 4 attributes:

  • firstname
  • lastname
  • age
  • isAlive

With the following respective values:

  • Roger [This is a string data-type and is therefore delineated by double quotes ' " ' ]
  • Moore [Again this is a string and needs double quotes
  • 89 [Number value that does not need quotes]
  • false [Boolean value, again does not need the double quotes]

Paste this JSON into something like jsoneditoronline.org, and you can interrogate its structure some more, (possibly not that useful for such a simple object...)

simplejson in editor

A, (slightly), more complex example

As mentioned in the overview of JSON, an object can contain "nested" objects, observe our person example above with a nested address object:

complexJSON

Here we can see that we have a 5th Person object attribute, address, which does not have a standard value like the others, but in fact contains another object with 3 attributes:

  • streetAddress
  • city
  • postcode

The values of all these attributes contains strings, so no need to labor that point further!

This nesting can continue ad-nauseum...

Again posting this JSON into our online editor yields a slightly more interesting structure:

complexjson in editor

A final example

Onto our last example which this time includes an array of phone number objects:

arrayJSON.png

Note: I removed: "age" and "isAlive" attributes from the person object as well as the "postcode" attribute from the address object purely for brevity.

You'll observe that we added an additional attribute to our Person object, "phonenNumbers", and unlike the "address" attribute it's contains an array of other objects as opposed to just a single nested object.

So What?

So why did I detail these JSON examples, we'll firstly it was to get you familiar with JSON and some of its core constructs, specifically:

  • The start and end of an object: ' { } '
  • Attribute-Value pairs
  • Nested Objects, (or objects as attribute values)
  • Array of Objects

Personally on my JSON travels these constructs are the main one's you'll come across, and as far as an introduction goes, should give you pretty good coverage of most scenarios.

Secondly, we'll be using these examples in deserializing JSON with JSON.Net, so lets's get started on that!

Installing JSON.NET

The best way to install JSON.Net into your Visual Studio project, (i think), is to use the NuGet Package Manager Console. So in your project, do the following:

NuGet

Once in the Package Manager Console, type:

PM> Install-Package newtonsoft.json

See below:

nugert success

You should also be able to see JSON.Net in the list of Project References:

jsonnet references

OK, We're Ready to Code

In order to deserialize a JSON string we're going to use the JsonConvert class from the JSON.Net framework, then use the DesrializeObject method, a link to the Newtonsoft web site can be found HERE, detailing this approach.

Let's be Dynamic

Ok our first example requires the least coding, first create a new method in your form's code called: deserialiseJSON, as follows:

 
private void deserialiseJSON(string strJSON)
{
  try
  {
    // TO DO: Our deserialize code
  }
  catch(Exception ex)
  {
    debugOutput("We had a problem: " + ex.Message.ToString());
  }
}

Now update the click event of our deserialize button:

 
private void cmdDeserialise_Click(object sender, EventArgs e) 
{ 
  deserialiseJSON(txtInput.Text); 
} 

This just means we're almost ready to start deserializing!

Go Back to our deserialiseJSON method, and in our try block, add the following code:

 
private void deserialiseJSON(string strJSON)
{
 try
 {
   var jPerson = JsonConvert.DeserializeObject<dynamic>(strJSON);
   
   debugOutput("Here's our JSON object: " + jPerson.ToString());
   debugOutput("Here's the First Name: " + jPerson.firstname);
 }
 catch(Exception ex)
 {
 debugOutput("We had a problem: " + ex.Message.ToString());
 }
}

So what have we done here?

  • Created a variable (var) called jPerson. This will hold our deserialised JSON object
  • Used the DeserializeObject method of the JsonConvert class
  • Used the "dynamic" object as a place holder for our "type" - more on this below
  • Passed in what ever serialized, "raw", json string is in our first text box

It's the use of dynamic that's most interesting here. This basically tells Visual Studio that we'll have to wait until run time before our object dependencies are resolved. In practice this means we can pass in any valid json string and the JSON/Net frameowrk will interpret and create a deserilaised c# object for us on the fly, (jPerson).

We can then access the attributes of jPerson, and treat it like a regular object.

So let's run our app, and pass in our simple json example:

examppl1

Success!

To Come

Still to write up the use of c# classes and supply those as our type.

 

 

Authenticating to a REST API from c#

Foreword

This tutorial follows on directly from my previous post: Consuming a REST API from c#. You may need to start there if you want to pick up the thread of the coding examples below.

 

A full video of this tutorial can be found here.

 

User Story

As a developer
I want to call a REST API that requires authentication
So that I can consume its “protected” resources

Acceptance Criteria

By the end of this tutorial you should be able to:

  • Authenticate to a REST API (using a c# Windows app), using Basic Authentication
  • Authenticate to a REST API (using a c# Windows app), using NTLM, (Windows), Authentication

Ingredients

For this tutorial you will need the following, (or something similar):

  • Windows PC (I’m running Windows 10)
  • Visual Studio (I’m using the 2017 Community Edition – which is free)
  • Web Browser (I’m using Firefox and Edge)
  • Access to a REST API that uses “Basic” Authentication
    • Don’t worry if you don’t we spin one up as part of this tutorial
  • Access to a REST API that uses “NTLM” Authentication
    • Don’t worry if you don’t we spin one up as part of this tutorial
  • ~40 Minutes of your time

My Lab Set Up

Having listed the ingredients above, here’s a simple schematic of my Lab Set up for this tutorial:

VP-6 REST Authentication My Lab

You’ll notice that I’m using the Firefox plugin called “Live HTTP Headers” this isn’t mandatory for the tutorial but it’s a useful little tool that allows you to see the HTTP Headers sent and received by Firefox. Kind of helps demystify the whole thing.

You’ll also notice that I have a Linux VM running “Jira”. This is absolutely not necessary but I included it as: a) I’ll be using it in a later tutorial, and b) it’s good to try this stuff against a proper production API.

What’s “Jira”?

Jira is a web-based tool from a company called Atlassian, it’s actually quite difficult to describe what it is, but in essence it’s an “Issue Tracker”. Think of it like a big “To Do List” repository for organsations. It’s used a lot by Technical help desks, (to raise fault / support / defect tickets), and software development teams to capture requirements and plan / track the release, (as well as bugs).

I actually use it to plan the tutorials & Youtube videos I’m going to produce, I even knew a guy who used it to plan his wedding!

I’ve included it as is provides a actual REST API interface that I can test against. You can safely ignore it if you want to though!

User Interface Design

Again using Balsamiq I created the following wire frame to help clarify my thinking on what the app is going to do. You’ll notice I’ve provided for:

  • username and
  • password credentials

which will be passed to our REST Class on an as needed basis. I won’t be covering how to  securely store credentials persistently as part of this tutorial. If that’s something of interest though drop me a line!

There’s also 2 radio button groups:

  1. Authentication Type
  2. Technique

Which I thought may be necessary at the start of this project to dynamically switch between, (surprise-surprise!), Authentication Type and Technique. As it transpires they’re probably a bit superfluous, but I’ve left them in for now.

VP-6 REST Authentication Wireframe

Class Diagram

We’re basically just extending upon the Class from the previous post, the updated diagram is shown below:

VP-6 REST Authentication Class DiagramI’ve added 2 new enumerations:

  • authenticationType
  • authenticationTechnique

Think they’re self explanatory, so we’ll move on…

Our constructor and makeRequest method have not changed, at least from a calling perspective, there are obviously some internal changes which is what we’re going to code up below.

We have some new attributes, again I think self-explanatory:

  • authType
  • authTech
  • userName
  • userPassword

Is that a good constructor?

You’ll notice, (when it comes to coding below), that the constructor doesn’t take any arguments and that we set all the class attributes, (e.g. authType etc.), following construction.

There is no real reason, (that I can think of), why this really needs to be the case, so obviously feel free to change that!

 

Coding – Part 1: Self Rolled “Basic” Authentication

Ok ok! We’re here finally at the coding – I’m guessing the bit you’re all interested in! Again to pick up this coding thread you should start here.

I’m going to assume that you’ve updated the UI with the following elements:

VP-6 Form Controls

So basically your UI will look like the following:

VP-6 REST Auth Form UI

Now we’re going to update our RestClient class as follows, (new code from the previous tutorial is in blue):

using System;
using System.IO;
using System.Net;


namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    public enum authenticationType
    {
        Basic,
        NTLM
    }

    public enum autheticationTechnique
    {
        RollYourOwn,
        NetworkCredential
    }

    class RestClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }
        public authenticationType authType { get; set; }
        public autheticationTechnique authTech { get; set; }
        public string userName { get; set; }
        public string userPassword { get; set; }


        public RestClient()
        {
            endPoint = string.Empty;
            httpMethod = httpVerb.GET;
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();

            String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
            request.Headers.Add("Authorization", authType.ToString() + " " + authHeaer);

            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
        }

    }

The really only interesting thing here is the addition of the String "authHeader", and we attach it to our HttpWebRequest object - "request".

What is this header string?

This "self-rolled" header string supports "Basic" Authentication - see the section below. We construct it so that it follows RFC2617 - The HTTP Basic Authentication scheme and pass it with our initial request so that we are authenticated through, (assuming the credentials are correct).

As per RFC2617, the string should be constructed using the following method:

  1. Obtain the user id and password (this will be supplied from our text boxes)
  2. Construct the "user-pass" be concatenating the user id, a single colon (":") character and the password. E.g. "john:abc123"
  3. Encode the "user-pass" into an octet sequence
  4. Encode the the octet sequence using Base64 into a sequence of US-ASCII characters

The spec goes into more detail, you can read it here.

All we need to do now is update out "Click" button event to pass the user name and password, (we can also set the Auth Type and Technique values here although they are not really used at this stage).

The updated click event handler will look something like this, (new code is in blue):

private void cmdGO_Click(object sender, EventArgs e)
        {
            RestClient rClient = new RestClient();
            rClient.endPoint = txtRequestURI.Text;
            rClient.authTech = autheticationTechnique.RollYourOwn;
            rClient.authType = authenticationType.Basic;
            rClient.userName = txtUserName.Text;
            rClient.userPassword = txtPassword.Text;

            debugOutput("Rest Client Created");

            string strResponse = string.Empty;

            strResponse = rClient.makeRequest();

            debugOutput(strResponse);
        }

If all is successful then this should allow you to use a REST API that uses Basic Authentication.

As an example I called my Jira REST API, (I wanted to retrieve the JSON for a single issue with id: wp-1), with the following request:

Note: I did not pass the any credentials over, and so I received the following response:

vp-6 unauthorised

You can see that Jira returned back a 401 response, (unauthorised). More about that later...

Trying again but this time supplying the correct credentials yields success and some JSON related to the issue:

vp-6 authorised

How Basic Authentication Works

There are plenty of resources on the internet that detail how Basic Authentication works, (I have placed a few links in the references section below), but thought I'd give a brief overview here as its relevant to a point I want to make...

The following sequence diagram illustrates a typical request response scenario when the initial request does not have the necessary authentication header:

VP-6 REST Basic Auth Sequence

You'll see that the 2nd request supplies and Authorisation Header as per RFC2617 and we are returned a successful 200 OK response.

This "challenge/response" scenario is important when we come to look at the NetworkCredential class as it relies on this to supply the credentials. I.e. it won't initially supply the credentials until it receives the 1st 401 Unauthorized response along with the Authentication type, in this case "Basic".

If it, (the NetworkCredential Class), does not receive this response, then it may not work correctly. This is actually the scenario I encountered with the Jira api, that being when I used the NetworkCredetial class with the Jira API I could not get it to work. Had I read the Jira API documentation I'd have realised that Atlassian, (the company that make Jira), did not implement their API that way, observe:

vp-6 jira doc

Why do you care?

Good question! The reason I have made a bit of a diversion is 2 fold:

  1. When we come to the NetworkCredential Class below it is useful background
  2. In researching this tutorial I have found many people having a similar issue with the NetworkCredential class, (when in fact it's not really the NetworkCredential class to blame, but API vendors that haven't implement the Basic Authentication standard correctly). Basically, (no pun intended), I don't want you to waste the same amount of time I did.

TIP

As boring as it sounds, READ your API's documentation. I may save you a lot of time!

Set Up Your own API?

The thought occurred to me that perhaps some people don't have access to a test api? Or in my case I wanted to spin up a really stock-standard api so I could do more testing and play with different types of authentication.

I was also confused why the Jira API did not work with NetworkCredential class so wanted to test with another Basic Authentication API I set up myself - as mentioned above had I read the API documentation my confusion would have dissipated!

I therefore decided to insert this section into the tutorial so I:

  • Could Demonstrate using the NetworkCredential Class
  • Play with NTLM, (Windows), authenticaton

How Many Types of Authentication?

There are a plethora of different authentication mechanisms in use, so would have been difficult for me to cover them all. I chose Basic and NTLM in this case.

Set Up IIS

Ok since we're running on Windows and as per my lab set up above, we're going to run a REST API using the Microsoft ASP .NET MVC framework, (don't worry if that doesn't make sense), which will require that we have IIS installed.

What About IIS Express?

We could use the embedded version of IIS, (IIS Express), that starts when you run your ASP.NET project from within Visual Studio, but I've found that doesn't give you the same power and flexibility when it comes to configuring the authentication options.

Step 1 - Open Control Panel
  • Click "Programs"

VP-6 Control Panel

Step 2 - Click: Turn Windows features on or off

vp-6 windows features

Step 3 - Select the IIS Features You Want, (or need!)

vp-6 iis

You'll notice I've placed arrows against the primary artifacts you'll need to run a REST API with authentication...

Click OK and assuming all's well the required components should be installed.

You can check that IIS is working ok by firing up a browser and navigating to "localhost": http://localhost, you should see something like this:

vp-6 iis welcome

Spin Up a Template API

OK cool so IIS is up an running, (hopefully), so all we need to do now is spin up a REST API... Here we go!

Step 1 : New Project
  • Open Visual Studio and select New Project
  • Expand the Visual c# node
  • Select "Web"
  • And Select "ASP .NET Web Application (.NET Framework)
  • Give it a name and hit OK

vp-6 New Project

Step 2: Select the "Web API" Template
  • Select the "Web API" Template
  • Ensure both "MVC" and "Web API" tick boxes are checked
  • DON'T Click OK YET! (We need to set authentication type)

vp-6 aps template.png

Step 3: Click "Change Authentication"

vp-6 change auth

Step 4: Select Windows Authentication
  • Select Windows Authentication (We can change this in IIS Manager later)
  • Click OK

vp-6 windows auth

Click OK once more, (to close out the ASP. NET Template Selection Window)

Your API Project will be created!

Step 5 - Edit the "Index" Method of the "Values" Controller

Don't worry if you don't underand what this is, we can cover it in another tutorial.

  • In the Solution Explorer, Expand the "Controllers" folder
  • Double Click the "ValuesController" Class file - the file should open in the editor
  • Optional: Change the "Value 1" and "Value 2" values in the 1st method to something else.
  • Edit the "Authorize" Directive at the top of the class to include a user and specified account. This will take the form: domain\username.

If you are running outside of a windows Domain, (if you're running a stand alone PC at home this will probably be the case), then the "domain" value is just your PC name.

Your Edited code should look like this:

vp-6 code

Note my PC name is: DESKTOP-H9580B0

My Account is: lesja

Step 6 - Build

I usually just to a build at this stage to make sure there are no errors:

  • Build Menu->Build Solution

vp-6 build

Deploy and Configure Authentication

Ok now we have our test API build - we can now deploy it to IIS:

Step 1: Right Click Project and Publish
  • Locate your "project" in solution explorer, (it should be the 2nd node under the main solution)
  • Right Click and select "Publish..."

vp-6 Publish

Step 2: Select IIS, FTP
  • Select the "IIS, FTP, etc" Option and click "Publish"

vp-6 Publish IIS

Step 3: Select File System Method
  • Select "File System" as the Publish Method in the resulting dialog box
  • Click on the "..." ellipsis for the target location and locate the folder you want to publish to.

vp-6 Publish to.png

Step 4 - Select Publish Destination
  • Create a New Folder if Required
  • Note that using this method the location you select should be on the same server where IIS is installed, (as we're doing all this on our PC it's fine)
  • Select the folder and Click OK

vp-6 Destination.png

  • Click Next

vp-6 next

  • Expand "File Publish Options"
  • Select "Delete all existing files prior to publish"
  • Click "Save"

 

vp-6 Publish settings2

 

  • The Project should be published to your chosen destination if all is well.
  • You should see something similar in your "Output" window:

vp-6 Publish Output

Step 5: Configure Application in IIS Manager
  • Type "IIS" into Cortana (if using windows 10) or just "Search Programs and Files" in older versions of Windows, (from memory you'll have to click the start menu)

vp-6 IIS

  • Internet Information Services (IIS) Manager should be found. (if it's not go back to the step where we installed IIS and ensure that you have the "Management Tools" box selected)

vp-6 IIS Menu

  • Once IIS Manager has started, expand the tree in the left hand plane until you see the "Default Web Site" node, (it looks like a little globe).
  • Right Click and Select "Add Application..."

vp-6 add application

  • Give the application an "Alias". I usually use the same name as the Project
  • Click the "..." ellipsis for the "Physical Path" destination and locate the folder location where you previously published the app, (see steps above).
  • Click OK

vp-6 app location

  • Expand, (if you haven't already), the "Default Web Site" node and you'll see your configured IIS "Application"

vp-6 new app

OK we're nearly there! All this hard work now means that we can configure the "Authentication" type for our app.

Step 6: Select The Applications Authentication Type

Note: You'll repeat this step multiple times as you switch between the 2 authentication types covered in this tutorial:

  • Baisc
  • NTLM (windows)

In IIS Manager, double-click on the "Authentication" Icon:

vp-6 Click Authentication

  1. Click on "Basic Authentication"
  2. Click Enable

vp-6 Set Basic

You should see that "Basic Authentication" is enabled for this application. You may also get a warning about SSL.

vp=6 baisc enabled

The point about SSL is extremely valid, if you're not using SSL, (essentially "https"), then the credentials you're sending over the wire are not secure...

Step 7: Test Site Manually in a Browser First

To just test that the basic site works, go to your favorite browser and navigate to the applications home page, this will be:

http://localhost/<application alias>

E.g. In my case:

http://localhost/tutorialRest

vp-6 Manual Browse to Home

We can now browse to our, (secured), API, which as you may remember formed part of the "Values Controller".

The URL you'll need is:

http://localhost/<application-alias>/api/values

So in my case this would be:

Assuming all is correct you'll get a 401 challenge from the server:

vp-6 401 challenge

As mentioned right at the start, (seems like a long time ago now right!), I installed a plugin to Firefox called "Live HTTP Headers", this allows us to look at, (surprise surprise), the http headers sent to and from IIS. Here's the output for that first request / response as shown above:

vp-6 http header 1

 

 

You'll notice that at this stage we hadn't sent the credentials...

Now following entering, (the correct!), credentials we get the following http header traffic:

vp-6 http header 2

And finally we get the following output in our Browser:

vp-6 http 200

Note: We can repeat step 6 - Select Application Authentication Type but this time for NTLM, (windows), authentication, (remembering to "disable" Basic Authentication first). If you do this and you interrogate the http header you'll see something like the following in the first response leg:

vp-6 http header 4

Step 8 - Test with Our C# Client

Ok, before we begin ensure that the test API has been set to use "Basic Authentication" once again!

  • Fire up our c# Rest Window client and make a first request to the test api, (don't supply any credentials you should see:

vp-6 Rest Client 401

You'll see that we correctly get a 401 response from the server.

  • Now let's enter our correct credentials and try again:

vp-6 Rest Client 200

And we have success!

We've basically called our test API, (that's using Basic Authentication), from our c# client by constructing our own header.

What happens with we switch the authentication type to NTLM, (windows), authentication and try again with our client?

vp-6 ntlm 401

yeah that's right we get a 401 response because we're still using a self-rolled authentication header that's used only by Basic Authentication.

What we need now is the "NetWorkCredential" Class!

Coding - Part 2: NetworkCredential

In our previous example we constructed our authentication header on our own, but we could have used a much simpler method - the NetworkCredential class, (assuming Basic Authentication is implemented as we expect).

To use the NetworkCredential class with both Basic and NTLM authentication it's pretty easy... Let's first update the code in our Form Click event so it sets the correct attributes on our RestClient object, (new code is in blue):

private void cmdGO_Click(object sender, EventArgs e)
        {
            RestClient rClient = new RestClient();
            rClient.endPoint = txtRequestURI.Text;
            
            if(rdoBasicAuth.Checked)
            {
                rClient.authType = authenticationType.Basic;
                debugOutput("authenticationType.Basic");
            }
            else
            {
                rClient.authType = authenticationType.NTLM;
                debugOutput("authenticationType.NTLM");
            }

            if(rdoRollOwn.Checked)
            {
                rClient.authTech = autheticationTechnique.RollYourOwn;
                debugOutput("autheticationTechnique.RollYourOwn;");
            }
            else
            {
                rClient.authTech = autheticationTechnique.NetworkCredential;
                debugOutput("autheticationTechnique.NetworkCredential");
            }


            rClient.userName = txtUserName.Text;
            rClient.userPassword = txtPassword.Text;

            debugOutput("REst Client Created");

            string strResponse = string.Empty;

            strResponse = rClient.makeRequest();

            debugOutput(strResponse);

Next we just add a simple if clause to our RestClient class:

  • If we want to use the "Self-Rolled" technique, then we default to basic authentication
  • Else if we want to use NetworkCredential Class then we let it take care of the Authentication Type (Basic or NTLM) - that's the power of using it!

Note when we come on to the CredentialCache we will use the attribute "authType".


using System;
using System.IO;
using System.Net;


namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    public enum authenticationType
    {
        Basic,
        NTLM
    }

    public enum autheticationTechnique
    {
        RollYourOwn,
        NetworkCredential
    }

    class RestClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }
        public authenticationType authType { get; set; }
        public autheticationTechnique authTech { get; set; }
        public string userName { get; set; }
        public string userPassword { get; set; }


        public RestClient()
        {
            endPoint = string.Empty;
            httpMethod = httpVerb.GET;
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();

            if (authTech == autheticationTechnique.RollYourOwn)
            {
                //We'll only do Basic Authentication if we roll our own
                String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
                request.Headers.Add("Authorization", "Basic " + authHeaer);
            }
            else
            {
                NetworkCredential netCred = new NetworkCredential(userName, userPassword);
                request.Credentials = netCred;
            }
            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
        }

    }

The only real code of interest is the NetworkCredential class its self:

  • We simply create a new instance of one, passing the username and password in the constructor
  • We then set the "credentials" attribute of our request object to our newly created NetworkCredential object

Lets' test with our client!

Test 1: API Set to Basic - Client Using Self-Rolled Header

vp-6 test1.png

Result: Pass!

Test 2: API Set to Basic - Client Using NetworkCredential Class

vp-6 test2

Result: Pass!

Test 3: API Set to Windows (NTLM) - Client Using Self Rolled Header

vp-6 test3

Result: Fail! - Technically this is not a fail as the software is behaving exactly as we expect, but just laboring the point that our self-rolled header does not allow us to authenticate when the api is using Windows authentication.

Test 4: API Set to Windows (NTLM) - Client Using NetworkCredential Class

vp-6 test4.png

Result: Pass! We have for the first time, authenticated to a Windows, (NTLM), protected API using the NetworkCredential class.

You can see the power and simplicity of this approach, but let's try one more test...

Test 5: JIRA API Set to "Basic" - Client Using NetworkCredential Class

As mentioned previously when using our Jira API - the self rolled approach worked fine, let's try with the NetworkCredential Class:

vp-6 test5

Result: Fail! - I actually would call this a fail this time. I think this should work. As mentioned above though, Atlassian did not implement their API according to the true nature of Basic Authentication.

Test 6: JIRA API Set to "Basic" - Client Using Self Rolled Header

vp-6 test6

Result: Pass! We knew this already though!

CredentialCache

I was going to write about the CredentialCache but the MSDN entry on it says everything I was going to say anyway! It can be found HERE.

In short:

  • Used to store multiple credentials
  • You assign it to HttpWebRequest Credentials attribute the same way we assigned the NetWorkCredential
  • When Adding multiple credentials you can assign the "Type" of authenticaton, e.g. Basic, NTLM etc, see below:

vp-6 Credential Cache

References

The unique reference for this tutorial is: VP-6

The Video

Consuming a REST API from c#

User Story

As a developer
I want to call a RESTful API from my c# application
So that I can consume and use the resources it provides

Overview

I’m guessing if you’re reading this that you probably have some idea what a RESTful API is, so I won’t delve too deeply into the why’s and wherefores, however some key points to note:

  • Centers around the concept of “resources” that you consume, create, update and destroy
  • Accessed via a “base” URI
  • Can return data in any format, (XML, atom etc.), but most usually JSON
  • Usually accessed over HTTP with the usual string of verbs: GET, POST PUT etc.
  • Platform agnostic
  • Lightweight (and usually fast!)

Ingredients

For this tutorial you will need the following, (or something similar):

  • Windows PC (I’m running Windows 10 on a Surface Pro 3)
  • Visual Studio Community Edition
  • Internet Access (if you want to use the sample REST service – if you’re reading this then I guess this is already sorted!)
  • 30-40 minutes of your time

Ok – Let’s get started!

Test the REST

Before we launch into writing code to access your REST service, you should simply be able to access it directly from your browser and observe the data returned, (in essence that’s really all this tutorial is – making a http request from c# code!).

Now as I’m a nice guy I’ve placed an example, (and very simple!), RESTful service on the interweb for you to play with, which coincidentally is the REST API I’m using for this tutorial.

Note: The API is read only, you’ll only be able to list all items in the db, or retrieve a particular item, (adding, updating and removing actions have all be disabled).

The API Can be found here:

http://dry-cliffs-19849.herokuapp.com/

This should take you to the “home-page” of the REST API, note that this is not really anything to do with the REST API its self, it’s just a nice landing page. (I say nice, the “styling” as you can see is straight out of the early 90s).

You will see a listing of all the REST Actions (and associated HTTP verbs), that will allow you to use our User resource.

HTTP Request URI Action Purpose
GET http://<base uri>/users index List all User resources
GET http://<base uri>/users/n show Get the user resource with an id of n
POST http://<base uri>/users create Create a new User resource
PATCH http://<base uri>/users/n update Update User resource with an id of n
DELETE http://<base uri>/users/n destroy Delete User resource with an id of n

The first 2 actions are activated and by clicking on the associated links you will be presented with the JSON response. For the first request, (listing all User resources), you should see something similar in your browser:

SNAGHTML250897

What you are seeing is a list of JSON objects representing a list of users in the database. Again not wanting to “cop out”, but I won’t go into too much detail about JSON, (I’m assuming if you’re reading this you have a basic understanding at least), however it’s really not that difficult…

Using an excellent tool like “jsoneditoronline.org”, you can cut and paste the JSON our test RESTService is returning to get a better idea of its anatomy, this will be useful for later:

SNAGHTML291e3e

As you see we were returned an array, (the ‘[‘ opening bracket denotes this), of user objects. Each object starts with a ‘{‘ and ends with a ‘}’.  Within the object we then have a list of key / value pairs, e.g. :

Key Example Value
id 3
name Adam Smith
email adam@email.com
nationality Scottish

It’s fairly easy now to understand the structure, and while it can get somewhat more complex in other instances, (e.g. when the value is another array of key value pairs), it’s essentially just the same design pattern. For the purposes of this tutorial it should suffice.

With all that done, (i.e. you’ve “tested” that you can access the REST service via your browser), it’s now time to start designing our solution.

User Interface Design

This section title is probably a little grandiose for what we are doing here, but I like to follow this pattern for even the simplest apps – that being mocking up the simple interface. I tend to think more visually so it’s only when I do this that I actually start to think how things will work…

The Mock Ups

I’ve used an excellent tool called “Balsamiq” to create the following mocks that depict the client before and after  making a call to the rest service. It is this interface we’ll want to construct in Visual Studio.

mocks

The eventuation of this amazing design will look something like:

actualclient

Sequence Diagram

Using some fairly rudimentary UML, the sequencing for the application looks as follows:

blog-11 sequence diagram

Probably the only thing of note is that our RESTClient, (i.e. our c# app), makes a synchronous call to our REST API… I.e. it waits for the response payload. This is probably not how you’d want to architect this in the real world, (you’d make a asynchronous request to allow for any network latency etc..). For the purposes of the tutorial it will suffice, and making an asynchronous request can be a bit of extra credit homework!

Again this is a very simple request response model, and possibly doesn’t require the need for a UML sequence diagram, however I think it’s just good practice, (especially when you are eventually faced with more complex sequencing!).

RESTClient Class Design

Ok now we’re getting down to it!

For the purposes of this tutorial we’re going to write a custom c# class that encapsulates functionality from multiple standard .net libraries. The class we’ll build is as follows:

Blog-11 RESTCLient Class

First we use an “enumeration” to map to our list of possible HTTP Verbs, you’ll see here we’ve only mapped GET & POST, and in fact we’ll actually only be using GET in this tutorial.

The actual class its self is pretty simple, containing 2 attributes:

  • endPoint (this is the URI we’re making the request to)
  • httpMethod (as describe above this just represents our HTTP Verb)

The class then has a very simple constructor and 1 additional method:

  • makeRequest (yes you’ve guessed it this initiates the request to our REST Api!).

Coding

Ok we’re finally here! It’s time to code.

Start Visual studio and create a new c# Windows Application – call it whatever you like.

Ok now we have a nice empty Windows form, you’ll want to place the following UI elements onto it:

  • TextBox to hold the Request URI (I called mine: txtRequestURI)
  • TextBox to hold the Response Payload (I called mine: txtResponse)
    • Set Multiline Property to true
    • Set ScrollBars Property to vertical
  • Button (I called mine: cmdGo)
  • Label
    • Set name Property to Request URI:
  • Label
    • Set name Property to Response:

With the form design complete, we can now move onto the creation of our RESTClient Class.

In Visual Studio right click your project, then select: Add -> Class..

snaghtml9c71c

When the Add New Item window appears, select Class (1), give it a name (2) – I called mine RESTClient.cs then click add (3):

untitled-picture

Your solution hierarchy should look somewhat like the following:

2016-09-30_12-37-24

Now add the following code to your RESTClient class, (or whatever you called it):

UPDATED: Used a Try / Catch / Finally Block instead to Using so we capture Non-successful responses.

using System;
using System.IO;            //Needs to be added
using System.Net;           //Needs to be added
using System.Text;          //Needs to be added

namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    class RESTClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }   

        //Default Constructor
        public RESTClient()
        {
            endPoint = "";
            httpMethod = httpVerb.GET;
            
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            var request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();
            
            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                //We catch non Http 200 responses here.
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
            
        }
    }
}
  •  The constructor basically sets our class attributes, (and as mentioned above we are defaulting to using the GET verb).
  • The makeRequest method creates a "request" object which is basically derived from the standard c# HttpWebRequest object, we then:
    • Make a Request
    • Wait for the response, (synchronous)
    • Check to see if there was an error
    • If no error then we parse the response and return it back to the caller as a string

Debug Output

Visual Studio has a range of tools to help you debug your code, (and they're excellent), but most of the time I like to include my own little function that writes out the contents of variables, or even just where I am in the code. I also use this function to write out things like our return string for the RESTClient class, feel free to use it it's pretty self explanatory:

  • It takes the string you want to write out, and send it to:
    • The Visual Studio Debug window
    • Any other text box you like, (in this example it's called txtResponse)
  • In order that the output "scrolls" to the end of the text box, you need to "ScrollToCaret"
private void debugOutput(string strDebugText)
{
  try
  {
    System.Diagnostics.Debug.Write(strDebugText + Environment.NewLine);
    txtResponse.Text = txtResponse.Text + strDebugText + Environment.NewLine;
    txtResponse.SelectionStart = txtResponse.TextLength;
    txtResponse.ScrollToCaret();
  }
  catch (Exception ex)
  {
    System.Diagnostics.Debug.Write(ex.Message, ToString() + Environment.NewLine);
  }
}

Wire Up the Button Click Event Handler

Finally we just need to create an instance of our RESTClient and make a call to makeRequest. To do this, simply go to the form designer and double click on the "GO!" button, this will create the default click event handler, then place the following code inside:

 private void cmdGo_Click(object sender, EventArgs e)
{
  RESTClient rClient = new RESTClient();

  rClient.endPoint = txtRequestURI.Text;
  debugOutput("RESTClient Object created.");

  string strJSON = string.Empty;

  strJSON = rClient.makeRequest();

  debugOutput(strJSON);

}

All it does is:

  • Instantiate a new instance of our RESTCLient
  • Set the end point to whatever we want to call
  • Prepare our response string
  • Make the call to the end point using our makeRequest function
  • Write the output to our text box, (and the Visual Studio debug window)

Give It A Go!

Run your project (F5) and try both of the following Request URI's:

In both cases you should get a response back from the REST service with HTML and JSON respectively.

2016-09-30_13-06-06

That's It!

Next Time...

Next time we'll tack the JSON response and work with it in c#

Push a New Git Repository to Github

Go to github and select “New Repository”

new repo

Enter the name of your repository (this needs to be unique to your github profile)

2016-04-13_21-17-47

Click “Create Repository”

Screen Shot 2016-04-13 at 8.50.38 PM

TIP: When you create a repo in Github, they basically give you the step by step instructions required at the local command line to initiate a new local repo and push it up to github:

2016-04-13_21-18-17

Go to your PC/Mac and fire up a command line…

Navigate into the directory of your software project:

2016-04-26_20-36-05

Initialise you local git repo (if there isn’t one).

git init

This command initiates a new local git db that is used for managing your source code, you should see a successful initiation message as detailed below:

gitinit

If you type the following, you'll see the current status of your local git repository:

git status

2016-05-03_20-57-18\

This shows us that while the repository has been initiated, none of the files (and directories), are being "tracked". This means they are not under git version control. To rectify this, issue the following command to add all files (and directories), for git tracking:

git add .

Get the status of your repository now and you will see that all files are being tracked:

2016-05-03_21-02-29

We now need to "commit" the changes we've made, (in this instance it's just adding all files to git version control), by issuing the following command, (the -m is message flag that allows us to add comments for this commit, this helps us track which files have been affected by a particular commit).

git commit -m "This is our first commit"

2016-05-03_21-07-42

Follow this with another status report on our repository (git status), and you should be told that there is nothing more to commit, and the working directory is clean...

At this stage our local git repository is up to date, (and if you were just a lone developer using git for version control you'd be "done"). However our local git repository is still not "associated" with the remote repository we created on github, (git and github are different things so don't confuse them!).

To make that association between your local git repository and the remote repository, type the following at your command line, (your's will look slightly different depending on the name of the repository you created on github):

git add remote origin https://github.com/binarythistle/aggregator.git

TIP: Remember the kind info the folks at github gave you when you created your github remote repository (see above)? Well this just the 2nd to last line.

Finally, issue the "push" command to push your local git repository to your newly associated github repository:

git push -u origin master

You'll be asked to authenticate to github when you perform a remote push...

Once the push is finished you're done! Both repositories, (local git and remote github are in sync).