Saturday, December 19, 2015

scanning for keywords as you type

If you need to detect the presence of keywords as you type, you can add a small amount of javascript to your client. This may be useful to detect the typing of tags and to populate your data model with structured data.
For example, if you want to detect football teams while you type, you could do something like this. You'll want to ensure your data model has been updated to have two-value Yes/No fields indicated by the uitests_ prefix.
// Provide a handler that can find hashtag keywords in a text box and update a data model. The keywords 
// are mapped to field names using a simple data structure. Everything here is hard-coded
// to help demonstrate the basic idea.

// We use methods that are easy to program for the POC but are not supported on all devices
// however, this is a way to do this for all devices, but let's be lazy this time.

// Mapping between hashtags (starting with #) and field names.
// Normally this is stored in a custom entity or uses some other scheme...
var mappings = {
"Ravens" : "uitests_Ravens",
"Patriots" : "uitests_Patriots",
"SF" : "uitests_SF"
};

// This could also be a call out to a web service but additional
// logic to make the call out efficient would be needed.
function attachChangeHandlers(controlName) {
    controlName = "description"
    var controlNameDOM = "description_i"

    // Process content scanning for hashtags.
    function processContent(content) { 
        for(hashtag in mappings) {
                if(!mappings.hasOwnProperty(hashtag)) { continue; }
                var attributeToSet = mappings[hashtag];            
                if(attributeToSet && content) {
                    if(~content.toLowerCase().indexOf("#" + hashtag.toLowerCase())) {                
                        Xrm.Page.getAttribute(attributeToSet.toLowerCase()).setValue(true);
                    }
                }
            }
    }


    // Normally we would define a namespace for our customizations...
    // This checks the map on every keystroke which is inefficient
    // but this can easily be optimized.
    // For 2016, can have the context passed in ;-)
    var keywordHandler = function() {
        try { 
            // CRM 2016 only! If you use 2015, you don't need domControl passed in.
            //var userInput = Xrm.Page.ui.controls.get(controlName).getValue();
            var userInput = this.value
            processContent(userInput)
        } catch(ex) {
            console.log(ex);
        }
        }


    // CRM 2016 has an API for key presses and getting the value of a control
    // as you type...use that after you upgrade because its more portable.
    //Xrm.Page.getControl(name).addOnKeyPress(keywordHandler)

    // The hard way...get the doc inside the MS CRM Dynamics IFrame.
    // parent.parent.document gets us to the topmost level.
    var doc = parent.parent.document.getElementById("contentIFrame0").contentWindow.document
    var inputControl = doc.getElementById(controlNameDOM);
    inputControl.addEventListener("keyup", keywordHandler);

    // Attach to the rich text editor as well
    window.setTimeout(function() { 
    var CKE = parent.parent.document.getElementById("contentIFrame0").contentWindow.CKEDITOR;
    if(CKE) {
        var richEditor = CKE.instances["new_richdescription_i"];
        if(richEditor) { 
            console.log("Found rich editor");
            richEditor.on('key', function() {
                processContent(richEditor.getData());
            });
        } else { 
            console.log("Did not find rich text editor!");
        }
    } else { 
        console.log("No CKEDITOR object found");
    }}, 2000);
}
Now in your form load event, call the setup function. As you type, the keywords are detected and the data model changed. Of course, you would usually hookup the keyword to field name mapping as an entity and pull it into the form when it loads, but that's an exercise for the reader.
The handler is attached to both a standard description box as well as a rich text description box (see my otehr blog). The delay is there with an overly generous delay to allow the CKEDITOR to load before the handler is attached. There's probably another way to do it, but most people will attach to the standard description box anyway so ignore that part of teh code.
MS CRM 2016 has API to allow attached key press handlers to controls, so some of the shanigans in this code are not needed in 2016 (yeah!).
With this setup, you can now detect twitter like hashtag words in text and change your data model e.g. "John mentioned that he likes #SF and the #Patriots" sets these attributes to true in the data model in real-time. This is convenient if someone is taking notes but they do not want to move their hands or finds tagging the record too burdonsome. There are "tag" oriented solutions you can add that add a nice tag model to your data model so you may want to look into those as well.

Saturday, December 5, 2015

Adding Rich Text Editor

Many people want a rich text editor in their CRM views to edit descriptions or notes. Solving this problem in general is hard across all devices such as desktop web and mobile, but here's a start.
First, look at this blog. It provides the general idea.
I had problems with this working for me, so I had to change the code to:
// Function that when called, loads the CKEditor from a CDN and converts some specially named fields...modify to meet
// the needs of your script block.

function convertToRichText() {

    // import the ckeditor script, but do it without web resources..  so we create a script tag in the DOM
    var headblock = parent.parent.document.getElementById("contentIFrame0").contentWindow.document.getElementsByTagName('head')[0];
    var newscriptblock = parent.parent.document.getElementById("contentIFrame0").contentWindow.document.createElement('script');
    newscriptblock.type = 'text/javascript';
    // we have to wait until the script is loaded before we can use it, so registering a callback event
    newscriptblock.onload = function() {

        // some configuration for the CKEDITOR rich text control
        var CKE = parent.parent.document.getElementById("contentIFrame0").contentWindow.CKEDITOR;
        CKE.config.allowedContent = true
        CKE.config.toolbarCanCollapse = true;
        CKE.config.toolbarStartupExpanded = false;
        CKE.config.width = '95%';

        var fieldsToReplace = ["new_richdescription"];
        for (var i = 0; i < fieldsToReplace.length; i ++) {
            var fieldname = fieldsToReplace[i];
            // We find the 'edit' control for the engagement overview and replace it with a rich text control
           var richtexteditor = CKE.replace(fieldname + '_i');

        richtexteditor.on('change', function() { 
          Xrm.Page.data.entity.attributes.get(fieldname).setValue(richtexteditor.getData());
        });
        richtexteditor.on('loaded', function ( field ) {
                // when the rich text control is done loading, we need to change the display so it shows the rich text - this covers the page load scenario
                $('#contentIFrame0', window.top.document).contents().find('div#' + field + ' div.ms-crm-Inline-Value').css('display', 'none');
                $('#contentIFrame0', window.top.document).contents().find('div#' + field + ' div.ms-crm-Inline-Edit').css('display', 'inline-block');
                $('#contentIFrame0', window.top.document).contents().find('div#' + field + ' div.ms-crm-Inline-Edit').css('width', '95%');

            }(fieldname));
        }
    };
    newscriptblock.src = '//cdn.ckeditor.com/4.5.5/standard/ckeditor.js';
    headblock.appendChild(newscriptblock);
}
And then it worked. However, I found it klunky so you need to ensure that you build a robust solution that works on all devices. You'll want to make this more robust in several ways but that's the gist of it.
Note that the Xrm toolkit recently released a rich text editor based on CKEditor. That's described here.

Friday, November 27, 2015

accessing the new web api service document ala odatav4 and scala with akka-http

If you need to access the root service document at the MS CRM Dynamics website, you can access it using standad OData techniques.
I typically use akka-http. Here's how you can obatin the service document at the start of your program. I'm not going to show all the related code but just enough to make it easier for you to access your service document.
...
val http = Http()
...
 val rootFuture = async {
      val response = await(http.singleRequest(HttpRequest(GET, "http://yourorg.crm.dynamics.com" + "/api/data", odataHeaders)))
      if(logger.underlying.isDebugEnabled()) {
        val body = Await.result(response.entity.toStrict(1 seconds).map(_.data.utf8String), 5 seconds)
        logger.debug(s"Service root document: $body")
      }
      await(Unmarshal(response).to[ServiceDocument])
    }
    val root = Await.result(rootFuture, 5 seconds)
I use async/await because any program should not proceed without first getting the root document then deriving all the URLs from it. Hence the use of awaits.
Here's I have defined a spray-json unmarshaller by defining a few case classes and then defining my own spray-json "formatter." Once you do this you can use the Unmarshal function (which is akka-http):
case class EntitySet(val name: String, val uri: Uri, val title: Option[String] = None, val kind: Option[String] = Some("EntitySet"))
case class ServiceDocument(val serviceRoot: Uri, val metadata: Uri, val entitySets: Seq[EntitySet]) {
  /**
   * Return the URI for obtaining an EntitySet.
   */
  def /(entitySetName: String): Option[Uri] = entitySets
    .find(_.name == entitySetName)
    .map(_.uri.resolvedAgainst(serviceRoot))
}

object OdataJsonProtocol extends SprayJsonSupport with DefaultJsonProtocol {

  implicit object UriReader extends JsonReader[Uri] {
    def read(value: JsValue): Uri = value match {
      case JsString(uri) => Uri(uri)
      case x@_ => throw new DeserializationException(s"Uri must be deserialized from a string not a $x")
    }
  }

  implicit object EntitySetFormat extends JsonReader[EntitySet] {
    def read(value: JsValue): EntitySet = value.asJsObject.getFields("name", "kind", "url", "title") match {
      case Seq(JsString(name), kind, JsString(url)) => EntitySet(name, uri = url, kind = kind.convertTo[Option[String]])
      case Seq(JsString(name), kind, JsString(url), JsString(title)) => EntitySet(name, uri=url, kind = kind.convertTo[Option[String]], title = Some(title))
      case x@_ => throw new DeserializationException(s"Unable to parse $x into an EntitySet")
    }
  }

  implicit object ServiceDocumentFormat extends RootJsonReader[ServiceDocument] {
    def read(value: JsValue): ServiceDocument = value.asJsObject.getFields("@odata.context", "value") match {
      case Seq(JsString(context), JsArray(values)) =>
        val entitySets = values.map { _.convertTo[EntitySet] }
        ServiceDocument(context.substring(0, context.indexOf("/$metadata")), context, entitySets)
      case _ => throw new DeserializationException("Error obtaining root document")
    }
  }
}
Notice the use of SprayJsonSupport. That allows normal spray-json to be used in the Marshal framework which is akka-http.
You'll also need some headers:
object OData {
  val odataHeaders = List(
    Accept(MediaTypes.`application/json`),
    RawHeader("OData-MaxVersion", "4.0"),
    RawHeader("OData-Version", "4.0"))
}
My odataHeaders are slightly modified to add the Bearer token (see my other blogs):
  val odataHeaders = List(Authorization(headers.OAuth2BearerToken(yourBearerToken))) ++ OData.odataHeaders

Wednesday, November 25, 2015

odata and scala - using odata java libraries in scala

Someone asked me about using odata java libraries with scala and other communication infrastructure such as dispatch, spray, akka-http or finangle.

You can use the Olingo (formerly SAP odata's library) or the SDL odata library with scala and use your own communication layer. Both libraries provide a communication layer if you want to use it.

However, both libraries have interconnecting sets of components to build up odata request content and parse response information. The communication layers in these two libraries use standard javax.net and java.net type mechanisms for communications.

Both libraries provide client side services:
  • A way to create queries. Queries in odata use query parameters such as $top or $select. The query objects in the libraries translate API into these query parameters on an HTTP request object.
  • A way to map java beans into odata entities. Typically this is done via annotations. You can think of this layer in the libraries similar to the way that an ORM needs to map jvm objects into a specific RDBMS repository put/get format.
  • A way to translate responses into java objects. For example, you can iterate over an entity set returned from an odata entity set request. The libraries provide API for the iteration that fits, somewhat, into standard java semantics.
Both libraries also allow you to create odata data sources, also known as the "server" side. Both libraries allow you to receive odata requests, for example for metadata, and create a metadata object that is automatically serialized as a response.

The real issue with the communication layer in these libraries is that they force a style of programming that is imperative and blocking. That's not an issue for many programs that you might write if they are simple and small enough (small in the sense of execution performance).

To a large degree, the libraries represent what is called "late bound" programming. Late bound programming means that you use annotations or other types of reflection programming to handle the java POJO translation process. Scala programmers would prefer something more typesafe and not use annotations since annotations in these libraries are not checked at compile time. You could take the CSDL description of the entities and generate "early bound" objects, but since the library only uses annotations and reflection, this does not buy much other than saving you some typing.

The libraries also have immature support for OAuth2. SDL odata seems to have some plumbing created but it does not support refresh tokens and timeout scenarios, so you'll have to roll your own. 

SDL odata does provide some scala support, but its mostly a map into the java side of the library. SDL includes an akka based server. It's not clear what state it is in, but it looks promising. The scala support in SDL appears to be for the server versus a focus on the client side.

Olingo is very heavy with a standard "factory-based" java API. Everything is a factory. The late binding approach means that entity "properties" are coded fairly laboriously e.g. you specify that an entity has a property that has a type that has a...and so on. Specific requests are typed so that they have a matching response object type. If you use these, then you can access the "members" of the response and it performs the underlying parsing of the REST odatav4 response body for you. That's all very nice. But it is very painful java syntax.

Tuesday, November 24, 2015

Accessing MS CRM Dynamics preview web api (CRM 2015 Update 1)...scala, dispatch, adal4j, etc.

I previously wrote about authenticating a native client app written in java against a Azure AD in order to access MS CRM Dynamics.
Once you authenticate, you can pass around your access token (using the OAuth2 model) and issues REST requests. The API uses the ODATAv4 protocol. ODATA is essentially what ODBC was to desktop computers in the 90s, a way to access data that is expressed in an object graph. The v4 specification finally gets things right by allowing navigation with links directly embedded in the returned data enabling a more robust hypermedia navigation style.
The actual entire protocol is fairly well documented here on MSDN. MS seems serious about allowing the polyglut programming world to finally access the CRM data.
We want to test our authentication. To do that, we'll first ask WhoAmI using the approach that Jason laid out in his java version located here.
We won't use the two leading ODATA libraries quite yet. The ODATA libraries (Apache Olingo and SDL ODATA) both assume an execution model in addition to a content composition model. We'll just use the scala library dispatch for execution. It uses a well proven and robust async HTTP library underneath called async-http-client.
Picking up from the last post on AD authentication for MS CRM. You can continue to use amm for your scala repl:
load.ivy("com.ning" % "async-http-client" % "latest.release")
load.ivy("net.databinder.dispatch" %% "dispatch-core" % "latest.release")
import dispatch._, Defaults._
If you need to refresh your authentication token, rerun two lines:
val resultFuture = context.acquireToken(resource, clientid, "username@yourdomain.onmicrosoft.com", "password", null)
val t = resultFuture.get
t.getExpiresOnDate
Now we can issue a request using the dispatch library:
val headers = Map("OData-MaxVersion" -> "4.0",
  "OData-Version" -> "4.0",
  "Accept" -> "application/json",
  "Authorization" -> ("Bearer " + t.getAccessToken))

val whoAmIRequest = (host("yourdomain.crm.dynamics.com").secure / "api" / "data" / "WhoAmI") <:< headers
val response = Http(whoAmIRequest OK as.String)
You can chain a handler onto the future so that when it completes, you print out the returned value. Or you could just wait a second (wall clock time) and then run
response.print
The userid, as a GUID, is in the response body.
If you want to get rid of having to add the headers to every request, you can define a filter:
// Setup the headers so they are automatically added
import com.ning.http.client.filter._
case class MSCRMFilter(val token: String) extends RequestFilter {
  def filter[T](ctx: FilterContext[T]): FilterContext[T] = {
    val newRequest = new RequestBuilder(ctx.getRequest)
    Map("OData-MaxVersion" -> "4.0",
      "OData-Version" -> "4.0",
      "Accept" -> "application/json",
      "Authorization" -> ("Bearer " + token)).foreach{ case (k,v) => newRequest.addHeader(k,v)}
    new FilterContext.FilterContextBuilder(ctx).request(newRequest.build).build
  }
}
// Setup a new AD aware HTTP client
val adhttp = Http.configure(_.addRequestFilter(new MSCRMFilter(t.getAccessToken)))
While this does not automatically handle the token timeout, it does automatically add the OAuth2 and OData headers to every request that goes out. Now we can issue the request without adding the headers:
val whoAmIRequest2 = host("yourdomain.crm.dynamics.com").secure / "api" / "data" / "WhoAmI"
val response2 = adhttp(whoAmIRequest2 OK as.String)
// Be lazy and check response2.isCompleted to ensure it completes :-) then print the result
response2.print
You will get the same response as before.
To get a list of accounts, you can issue another OData like request:
val crmhost = host("yourdomain.crm.dynamics.com").secure // setup a reusable host value
val accountRequest = (crmhost / "api" / "data" / "accounts") <<? Map("$select" -> "name", "$top" -> "20")
val accounts = adhttp(accountRequest OK as.String)
accounts.print
Or obtain the contacts:
val contactsRequest = (crmhost / "api" / "data" / "contacts") <<? Map("$top" -> "20")
val contacts = adhttp(contactsRequest OK as.String)
accounts.print
But you'll notice that you will want to specify more query parameters in order to customize the ODATA response and not just obtain foreign keys as values for some af the properties, but the actual object content. You'll neeed to look at the ODATA documentation for details on how to pull more values from related entities in the same query.
In the above URLs we are violating good design. Because ODATAv4 follows a much more adherent design to REST, you can navigate from the initial metadata and entity list to any object without ever having to construct the actual URL path. You will need to add query parameters and more headers to cusomize the ODATA request, but you should never really have to spell out the paths like we did above.