Getting .NET Core Web API to run on Ubuntu Raspberry Pi

These steps have been tested on a Raspberry Pi with Linux Ubuntu 16.04. The Web API project is created on Windows 10 machine.

Prepare Windows development machine

* Install Install .NET Core 2.0 SDK from https://github.com/dotnet/cli/tree/master
After installation check the version by running the following dotnet command in a Command Prompt.
dotnet --info
Output will be like:
.NET Command Line Tools (2.0.0-preview1-005448)

Product Information:
Version:            2.0.0-preview1-005448
Commit SHA-1 hash:  f8c3c4a030

Runtime Environment:
OS Name:     Windows
OS Version:  10.0.14393
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\2.0.0-preview1-005448\

* Add development nuget sources by adding key values to the nuget.config. Nuget will now be able to get the latest packages that are under development:
Open C:\Users\xxxx\AppData\Roaming\NuGet\Nuget.config

add the keys:
<add key="aspnetcore-dev" value="https://dotnet.myget.org/F/aspnetcore-dev/api/v3/index.json" />
<add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
<add key="dotnet-corefxlab" value="https://dotnet.myget.org/F/dotnet-corefxlab/api/v3/index.json" />
<add key="aspnetcore-ci-dev" value="https://dotnet.myget.org/F/aspnetcore-ci-dev/api/v3/index.json" />

Prepare Raspberry Pi

* Download Ubuntu Mate 16.04 from https://ubuntu-mate.org/raspberry-pi/ and install it on your Raspberry Pi.

* Install all of the required libraries for .NET Core:
sudo apt-get install libunwind8 libunwind8-dev gettext libicu-dev liblttng-ust-dev libcurl4-openssl-dev libssl-dev uuid-dev unzip

* Download and install .NET Core runtime components for Ubuntu 16.04 for ARM 32 (check: https://github.com/dotnet/core-setup#daily-builds). You should be able to do:
wget https://dotnetcli.blob.core.windows.net/dotnet/master/Binaries/Latest/dotnet-ubuntu.16.04-arm.latest.tar.gz
sudo mkdir -p /usr/share/dotnet
sudo tar -xvf dotnet-ubuntu.16.04-arm.latest.tar.gz -C /usr/share/dotnet/
sudo ln -sf /usr/share/dotnet/dotnet /usr/bin/dotnet

* determine dotnetcore runtime version by running command 'dotnet --info'
I my case it will show a version of 2.0.0-beta-001791-00
We will use this version information later on in the process.

* create directory ‘mywebapi’ in your home directory. In later step we will publish the files to this directory.

Create Web API Project on development machine

* create a .NET Core Web API project name ‘mywebapi’ by running command: dotnet new webapi -n mywebapi
Project file will look like:
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp1.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Folder Include="wwwroot\" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore" Version="1.1.1" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="1.1.2" />
    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="1.1.1" />
  </ItemGroup>
</Project>

* Modify project file to:
- enable publishing for ubuntu 16.04. Add <RuntimeIdentifiers>ubuntu.16.04-arm</RuntimeIdentifiers>
- target .NET Core 2.0. Change TargetFramework into <TargetFramework>netcoreapp2.0</TargetFramework>
- make sure packages and publish will be for the .NET Core framework running on Raspberry Pi. In my case this was 2.0.0-beta-001791-00, so add node <RuntimeFrameworkVersion>2.0.0-beta-001791-00</RuntimeFrameworkVersion>
- add reference to make sure the libvu.so is being deployed. This was previously part of on of the standard Microsoft.AspNetCore package, however it was remove. So a reference to the libuv package. Add: <PackageReference Include="Libuv" Version="1.10.0-preview1-22036" />

The final project file:
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <RuntimeFrameworkVersion>2.0.0-beta-001791-00</RuntimeFrameworkVersion>
    <RuntimeIdentifiers>ubuntu.16.04-arm</RuntimeIdentifiers>
  </PropertyGroup>

  <ItemGroup>
    <Folder Include="wwwroot\" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore" Version="1.1.1" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="1.1.2" />
    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="1.1.1" />
    <PackageReference Include="Libuv" Version="1.10.0-preview1-22036" />
  </ItemGroup>
</Project>

* Get the latest packages by using the dotnet restore command
dotnet restore

* Publish the Web API project for Ubuntu by using –r and the RuntimeIdentifier that was in the project file.
dotnet publish -r ubuntu.16.04-arm
This command published files to the \bin\Debug\netcoreapp2.0\ubuntu.16.04-arm\publish\ folder.

* copy files from build output (\bin\Debug\netcoreapp2.0\ubuntu.16.04-arm\publish) to the Raspberry Pi.
You can used pscp (that is available when Putty is installed). Open Command Prompt, go to \bin\Debug\netcoreapp2.0\ubuntu.16.04-arm\ folder. Run the following command to copy all files (including all (sub)directories) to the Raspberry Pi into the mywebapi directory.
pscp -l <raspberry-loginname> -scp -r publish <ip-address>:mywebapi

Start the Web API application on Raspberry Pi

* On Raspberry Pi go to the mywebapi directory. Run 'dotnet mywebapi.dll' to start the Web API application.
Output:
Hosting environment: Production
Content root path: /home/someusername/publish/publish
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.

* To test the Web API service do a wget to ValuesController that is part of the sample Web API project.
wget http://localhost:5000/api/values

it will create a file values containing the response, in this case:
["value1","value2"]

How to authenticate your Android App to SharePoint 2013

This blog post shows how to authenticate against an on premise SharePoint 2013 environment and do a simple REST call to retrieve the title of the root web. The SharePoint environment is configured to authenticate with Forms Based Authentication and runs on https.

Various other posts about an Android app authenticating to SharePoint are using a WebView solution. But I didn't want the user to use a integrated web view, but just do the authentication in code. The main point is to get hold of the FedAuth token from a cookie and pass the token back into all calls to SharePoint.

The main steps to authenticate and do a REST call to SharePoint are as follows:
1- Configure the CookieManager
2- Authenticate user: Start a AsyncTask to do a SOAP request to https://sharepoint.dev/_vti_bin/authentication.asmx
3- The cookie returned in step 2 will contain the FedAuth token.
4- the REST call: Start a HttpURLConnection to do a GET request to https://sharepoint.dev/_api/web/title
     Make sure this HttpURLConnection contains the FedAuth token in Cookie.
5- Parse JSON result

Now some more details please…

1- Configure the CookieManager
In Frament: onCreateView. Create a CookieManager that handles cookies within the VM. Make sure to set it to default. The network calls will use this CookieManager. That includes the CookieStore which will contain the cookies received by network calls.

   1: CookieHandler cookieHandler = CookieHandler.getDefault();

   2: if (cookieHandler==null) {

   3:     CookieManager cookieManager = new CookieManager();

   4:     CookieHandler.setDefault(cookieManager);

   5: }

2- Authenticate user
Doing a SOAP call to  https://sharepoint.dev/_vti_bin/authentication.asmx in AsyncTask, doInBackground.

Line 1-4: various SOAP settings
Line 6/7, 12/13: username and password
Line 24: doing the SOAP call

   1: String SOAP_ACTION1 = "http://schemas.microsoft.com/sharepoint/soap/Login";

   2: String NAMESPACE = "http://schemas.microsoft.com/sharepoint/soap/";

   3: String METHOD_NAME1 = "Login";

   4: String AuthURL = "https://sharepoint.dev/_vti_bin/authentication.asmx";

   5:  

   6: String username = "koenvanderlinden";

   7: String password = "P@ssw0rd";

   8:  

   9: SoapObject request = new SoapObject(NAMESPACE, METHOD_NAME1);

  10:  

  11: //insert username and password to SoapEnvelope

  12: request.addProperty("username", username);

  13: request.addProperty("password", password);

  14:  

  15: //Declare the version of the SOAP request

  16: SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);

  17:  

  18: envelope.setOutputSoapObject(request);

  19: envelope.dotNet = true;

  20:  

  21: try {

  22:     HttpTransportSE androidHttpTransport = new HttpTransportSE(AuthURL);

  23:  

  24:     // do the SoapCall

  25:     androidHttpTransport.call(SOAP_ACTION1, envelope, null);

  26:     // At this point the CookieManager will containt a cookie that has the FedAuth token.

  27:     

  28: } catch (Exception e) {

  29:     Log.v(LOG_TAG, e.toString());

  30:     e.printStackTrace();

  31: }

3- Inspect cookies
After the call androidHttpTransport.call() (line 22) the CookieManager will contain the cookies, including the FedAuth token.
To inspect the cookies add the following lines:

   1: // check cookies

   2: CookieManager cookieManager = (CookieManager)CookieHandler.getDefault();

   3: List<HttpCookie> cookies = cookieManager.getCookieStore().getCookies();

 
4- the REST call:

Line 4: Set REST url to get web title
Line 7: Create HttpUrlConnection
Line 10: Add application/json;odata=verbose to “Accept” header to get response in JSON format.
Line 11: Start the HttpURLConnection to do a GET request

The HttpURLConnection will use the CookieManager that is set as default. The CookieManager contains the CookieStore, which contains all cookies. Because the HttpURLConnection is doing a request to domain sharepoint.dev, the cookies will be added to the request that match on that domain. In this case the FedAuth cookie matches and is added to the request.

   1: HttpURLConnection urlConnection;

   2: BufferedReader reader;

   3:  

   4: Uri uri = Uri.parse("https://sharepoint.dev/_api/web/title");

   5: URL url = new URL(uri.toString());

   6:  

   7: urlConnection = (HttpURLConnection) url.openConnection();

   8: urlConnection.setRequestMethod("GET");

   9: // we need JSON formatted result

  10: urlConnection.setRequestProperty("Accept", "application/json;odata=verbose");

  11: urlConnection.connect();

  12:  

  13: // Read the input stream into a String

  14: InputStream inputStream = urlConnection.getInputStream();

  15: StringBuffer buffer = new StringBuffer();

  16: if (inputStream == null) {

  17:     // Nothing to do.

  18:     return null;

  19: }

  20: reader = new BufferedReader(new InputStreamReader(inputStream));

  21:  

  22: String line;

  23: while ((line = reader.readLine()) != null) {

  24:     // Since it's JSON, adding a newline isn't necessary (it won't affect parsing)

  25:     // But it does make debugging a *lot* easier if you print out the completed

  26:     // buffer for debugging.

  27:     buffer.append(line + "\n");

  28: }

  29:  

  30: if (buffer.length() == 0) {

  31:     // Stream was empty.  No point in parsing.

  32:     return null;

  33: }

  34:  

  35: // JSON result

  36: String restResult = buffer.toString();

5- Parse JSON result

   1: // parse JSON result

   2: JSONObject jsonResult = new JSONObject(restResultString).getJSONObject("d");

   3: title = jsonResult.getString("Title");

Additional Information
This code makes use of the ksoap2-android library. The Gradle configuratoin has been changes to include ksoap2 library by adding the following parts:

Add: repositories to android.buildTypes
        repositories {
            maven { url 'https://oss.sonatype.org/content/repositories/ksoap2-android-releases/' }
        }
Add: compile 'com.google.code.ksoap2-android:ksoap2-android:3.1.1' to dependencies.

Sample gradle:

   1: apply plugin: 'com.android.application'

   2:  

   3: android {

   4:     compileSdkVersion 23

   5:     buildToolsVersion "23.0.1"

   6:  

   7:     defaultConfig {

   8:         applicationId "nl.idoconsultancy.androidauthsp"

   9:         minSdkVersion 21

  10:         targetSdkVersion 22

  11:         versionCode 1

  12:         versionName "1.0"

  13:     }

  14:     buildTypes {

  15:         release {

  16:             minifyEnabled false

  17:             proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'

  18:         }

  19:         repositories {

  20:             maven { url 'https://oss.sonatype.org/content/repositories/ksoap2-android-releases/' }

  21:         }

  22:     }

  23: }

  24:  

  25: dependencies {

  26:     compile fileTree(dir: 'libs', include: ['*.jar'])

  27:     compile 'com.android.support:appcompat-v7:23.0.1'

  28:     compile 'com.android.support:design:23.0.1'

  29:     compile 'com.google.code.ksoap2-android:ksoap2-android:3.1.1'

  30: }

Operating your Hue remotely using Hue Remote API – a working Proof of Concept

Introduction

To operate your Hue system on local network you can send REST based messages like described in the Philips hue API to your Hue bridge. However if you want to do this in your custom application over the internet you have a problem. No API, no official documentation about how to sent messages through the Hue Portal that forwards it to your Hue bridge.

This blog post will show how you can send messages to your Hue bridge over the internet (without using port forwards). In this proof of concept I reuse the access token of IFTTT. The code samples are in C#, however you could make the same REST calls using the language of your choice.

BTW: I assume you have an account on the Hue portal and that you registered your Hue bridge.

What others write about this topic

Just a few people have some information about this like:
http://blog.paulshi.me/technical/2013/11/27/Philips-Hue-Remote-API-Explained.html
Hacking Lightbulbs: Security Evaluation of the Philips Hue Personal Wireless Lighting System by Nitesh Dhanjani
http://stackoverflow.com/questions/19900657/how-to-connect-backend-service-with-philips-hue-bridge-remotely

I investigated some of the findings from above resources. The parts about the getting the access token is not working, it’s probably out of date because of changes to the Hue Portal. So if you don’t have an access token you cannot send commands to the Hue Portal.

Local REST Messages

Sending a REST based message to your local bridge is done by sending a http request in the following format.

URL	http://<your bridge ip>/api/<username>/<some resource>
Method	GET/PUT/POST/DELETE
Body	JSON formatted data

To turn on light 1 you will send a message like:

URL	/api/<username>/lights/1/state
Method	PUT
Body	{ “on”: true }

Remote REST Message

Now it gets interesting. Sending a message to your bridge through the Hue Portal is a bit different. You should always use POST and send the request to the Hue portal. It should also include a valid access token. The body contains a clipmessage which contains a clipCommand that encapsulates the JSON command you normally sent to your bridge (in local network mode).

The format is as follows:

URL	https://www.meethue.com/api/sendmessage?token=<AccessToken>
Method	always POST
Body	clipmessage=<JSON formatted clipCommand> clipCommand==> {    “clipCommand” :          {             “url”: “api/0/<some resource>”,             “method”: “GET/PUT/POST/DELETE”,             “body”: JSON formatted data          } }
Header	encoding: UFT-8 application/x-www-form-urlencoded

Note: the url in clipCommand uses ‘0’ what in a local scenario would be the username.

Below an example of a command that will turn on light 1.

URL	https://www.meethue.com/api/sendmessage?token=<AccessToken>
Method	POST
Body	clipmessage= {     “clipCommand” :          {             “url”: “/api/0/lights/1/state”,             “method”: “PUT”,             “body”:                { “on”: true }          } }

To retrieve the status of the Hue system you need to sent a message as follows:

URL	https://www.meethue.com/api/getbridge?token=<AccessToken>
Method	GET

The result will be a JSON string representing the current state of your Hue system.

AccessToken

The biggest challenge in all this is to obtain a valid access token. In the next step I’m going to get the access token that IFTTT is using. But you could also use the access token of the official Philips Hue app.

Steps:

1- Go to IFTTT

2- Search for Channel ‘Philips Hue’

3- Activate Channel, this will bring you to the Hue Portal and log you in, and ask you to trust IFTTT. Click ‘Yes’. The Hue Portal will generate an access token and IFTTT will use that access token to send command to your Hue bridge.

4- When channel is activated logon to Hue Portal

5- Go to ‘Settings’

6- Click on ‘My Apps’ and you will see a list of apps you trusted for accessing your Hue bridge.

7- Copy the link of the ‘(De-activate)’ link. It will contain the access token. Here’s an example of my token: https://www.meethue.com/en-us/user/preferencesappsrevoke?tokenId=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaTT0%3D
The access token you should send in the remote message should be: “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaTT0=” (without the double quotes and make sure to URL encode the whole thing).

BTW: If you revoke the access token by clicking ‘De-activate’ the access token cannot be used anymore. You will get a JSON response containing the message “I don't know that token”.

Code samples

We now have the message structure and the access token. We can now put this together in a C# sample.

   1: dynamic command = new ExpandoObject();

   2: command.clipCommand = new

   3: {

   4:     //change state of light 1

   5:     url = "/api/0/lights/1/state",

   6:     method = "PUT",

   7:     body = new

   8:     {

   9:       //set the light state changes

  10:       on = true

  11:     }

  12: };

  13:  

  14: SendMessage(command).Wait();

Lines 1-12 builds the clipCommand.
Line 14: sends the command

   1: private static async Task SendMessage(dynamic command)

   2: {

   3:   string jsonMessage = JsonConvert.SerializeObject(command);

   4:  

   5:   HttpClient client = new HttpClient();      

   6:   HttpContent content = new StringContent("clipmessage=" + jsonMessage, Encoding.UTF8, "application/x-www-form-urlencoded");

   7:  

   8:   Uri remoteMessageUrl = new Uri("https://www.meethue.com/api/sendmessage?token=<YOUR ACCESS TOKEN>");

   9:   var result = await client.PostAsync(remoteMessageUrl, content).ConfigureAwait(false);

  10:   var jsonResult = await result.Content.ReadAsStringAsync().ConfigureAwait(false);  

  11: }

Line 3: creates JSON formatted string of the command.
line 6: creates the body of the message, including the encoding and mediatype
line 8: creates URI including the access token
Line 9: sends request to Hue Portal
line 10: will contain results of the request.
Possible results could be:
success: "{\"code\":200,\"message\":\"ok\",\"result\":\"ok\"}"
failure: "{\"code\":109,\"message\":\"I don\\u0027t know that token.\",\"result\":\"error\"}"
failure: "{\"code\":113,\"message\":\"Invalid JSON.\",\"result\":\"error\"}"

 

Show me the Code

Get the C# sample from https://github.com/koenvanderlinden/RemoteHueDemo that contains sample for sending a message to change lights and how to retrieve bridge status.
Let me know if you think this blog post helped you in building a Hue enable application ;-).

TODO: getting the access token is still done by hand and it reuses another popular Hue application. Next step will be how to make the Hue Portal trust your own third party Hue app so you can use your own access token.

SharePoint issue “The context has expired and can no longer be used. Exception from HRESULT 0x80090317”

Problem
A client had problems with some workflow instances. Most instances did complete without a problem others failed.  Especially workflow that continued after a few days reported the following error in the workflow history log:
“{Microsoft.SharePoint.SPException: The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317) ---> System.Runtime.InteropServices.COMException (0x80090317): The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)
   at Microsoft.SharePoint.Library.SPRequestInternalClass.SetHttpParameters(String bstrHttpMethod, String “

I did some googling and found some hits that found a solution in synchronizing the date/time of the WFE’s and admin servers. In our case that wasn’t the the case.

Cause
After digging into the problem I noticed particular workflow activity opened a site collection using an SPUserToken.
using (SPSite site = new SPSite(WorkflowProperties.SiteId, InitializationData.UserToken))
{
//do some work
}
The SPUserToken was part of the workflow context and was populated when the workflow instance was first created. Default the SPUserToken is valid for 1440 minutes (1 day). In case the workflow instance is de-hydrated after more than 1 day the SPUserToken is expired and isn’t valid anymore. You will get the message “The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317” and are not allowed to open the sitecollection.

Conclusion
So when you get this specific error make sure to check if you use an old SPUserToken! And be aware the the SPUserToken can expire.

Detailed exception information
Full error message: "The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)"
Sample Stacktrace:
{Microsoft.SharePoint.SPException: The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317) ---> System.Runtime.InteropServices.COMException (0x80090317): The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)
   at Microsoft.SharePoint.Library.SPRequestInternalClass.SetHttpParameters(String bstrHttpMethod, String bstrRequestDigest, UInt32 flags, Guid gTranLockerId, Byte[]& ppsaImpersonateUserToken, Boolean bIgnoreTimeout, String bstrUserLogin, String bstrUserKey, UInt32 ulRoleCount, String bstrRoles, Boolean bWindowsMode, ApplicationPrincipalInfo& pAppUserInfo, Boolean bInvalidateCachedConfigurationProperties, Int32 lAppDomainId, ISPManagedObjectFactory pFactory, Boolean bCallstack)
   at Microsoft.SharePoint.Library.SPRequest.SetHttpParameters(String bstrHttpMethod, String bstrRequestDigest, UInt32 flags, Guid gTranLockerId, Byte[]& ppsaImpersonateUserToken, Boolean bIgnoreTimeout, String bstrUserLogin, String bstrUserKey, UInt32 ulRoleCount, String bstrRoles, Boolean bWindowsMode, ApplicationPrincipalInfo& pAppUserInfo, Boolean bInvalidateCachedConfigurationProperties, Int32 lAppDomainId, ISPManagedObjectFactory pFactory, Boolean bCallstack)
   --- End of inner exception stack trace ---
   at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)
   at Microsoft.SharePoint.Library.SPRequest.SetHttpParameters(String bstrHttpMethod, String bstrRequestDigest, UInt32 flags, Guid gTranLockerId, Byte[]& ppsaImpersonateUserToken, Boolean bIgnoreTimeout, String bstrUserLogin, String bstrUserKey, UInt32 ulRoleCount, String bstrRoles, Boolean bWindowsMode, ApplicationPrincipalInfo& pAppUserInfo, Boolean bInvalidateCachedConfigurationProperties, Int32 lAppDomainId, ISPManagedObjectFactory pFactory, Boolean bCallstack)
   at Microsoft.SharePoint.SPGlobal.CreateSPRequestAndSetIdentity(SPSite site, String name, Boolean bNotGlobalAdminCode, String strUrl, Boolean bNotAddToContext, Byte[] UserToken, String userName, Boolean bIgnoreTokenTimeout, Boolean bAsAnonymous)
   at Microsoft.SharePoint.SPWeb.InitializeSPRequest()
   at Microsoft.SharePoint.SPWeb.InitWebPublic()
   at Microsoft.SharePoint.SPWeb.get_CurrentUser()