Monday, March 12, 2018

programmatically setting settings

programmatically setting settings

With Dynamics CRM online, you have never been able to automate a solution deployment fully. With the on-prem version, you can modify the underlying database as needed to accomplish certain tasks but you do not have access to the database in the online version.

If you want to “script” the settings for an org that you just created, you were a bit out of luck. You can export/import a limited set of settings along with your solution, however, not all settings can be set using the solution import/export model.

For example, I have some large web resources that need to be uploaded in a solution. You cannot set export/import the attachment max file size setting, which controls the allowable size of web resources, using the web API or the solution export/import model. This is a well known problem.

How can you automate this?

Use the new Web API in v9.0! The new API allows you to set the settings (except for some settings around categorized search and the preview APIs).

The new web API has an entity called “organization” that contains most, but not all, of the settings.

Use it like any other web API call.

You can always try out the command line tool dynamics-client which allows you set the settings from a file. You can also use powershell but I found it easier to create a json settings file and just have that loaded for me vs writing a script.

Saturday, January 20, 2018

microsoft dynamics crm and electron web app.md

microsoft dynamics crm and electron web app.md

You can easily create a desktop app using javascript, css and html. The electron platform app allows you to create a javascript app, just like you would with the web version of dynamics crm and place it on the desktop. The only real difference is that you are not using crm forms. It’s a custom app.

Here’s a recipe for doing it:

  • create your web app. In my case, I have a “tool” that I use that typically loads in as a dynamics solution. I’m going to modify only a few lines to get it to work with electron. The tool shows plugin trace logs and updates itself using polling to track the latest log.
  • node
  • electron
  • react
  • office-ui-fabric-react

Modify Your App Entry Point

I use the dynamics-client-ui client ui toolkit for creating react based interfaces for dynamics crm. The other code has been touched to include some electron specific init code:

import * as React from "react"
import * as ReactDOM from "react-dom"
import * as PropTypes from "prop-types"
const cx = require("classnames")
const fstyles = require("dynamics-client-ui/lib/Dynamics/flexutilities.css")
const styles = require("./App.css")
import { Fabric } from "office-ui-fabric-react/lib/Fabric"
import { PluginTraceLogViewer } from "./PluginTraceLogViewer"
import { Navigation } from "./Navigation"
import { Dynamics, DynamicsContext } from "dynamics-client-ui/lib/Dynamics/Dynamics"
import { getXrm, getGlobalContext, isElectron } from "dynamics-client-ui/lib/Dynamics/Utils"
import { XRM, Client, mkClientForURL } from "dynamics-client-ui"
import { BUILD, DEBUG, API_POSTFIX, EXEC_ENV } from "BuildSettings"
import "dynamics-client-ui/lib/fabric/ensureIcons"
import { Config, fromConfig } from "dynamics-client-ui/lib/Data"

let config: Config

if (EXEC_ENV === "ELECTRON") {
    console.log("Configuring data access for electron")
    const electron = require("electron")
    const tokenResponse = electron.remote.getGlobal("adalToken")
    const adalConfig = electron.remote.getGlobal("adalConfig")
    if (!tokenResponse || !adalConfig) console.log("Main vars were not passed through correctly. tokenResponse:", tokenResponse, "adalConfig:", adalConfig)
    config = { APIUrl: adalConfig.dataUrl, AccessToken: () => tokenResponse.accessToken }
} else {
    console.log("Configuring data access assuming in-server web")
    config = { APIUrl: getGlobalContext().getClientUrl() }
}

export interface Props {
    className?: string
    config: Config
}

export class App extends React.Component<Props, any> {
    constructor(props, context) {
        super(props, context)
        this.client = fromConfig(props.config)
    }
    private client: Client

    public static contextTypes = {
        ...Dynamics.childContextTypes,
    }

    public render() {
        return (
            <div
                data-ctag="App"
                className={cx(fstyles.flexHorizontal, styles.app, this.props.className)}
            >
                {false && <Navigation className={cx(fstyles.flexNone, styles.nav)} />}
                <PluginTraceLogViewer
                    client={this.client}
                    className={cx(styles.plugin, fstyles.flexAuto)}
                />
            </div>
        )
    }
}

export function run(el: HTMLElement) {
    ReactDOM.render(
        <Fabric>
            <App
                className={cx(styles.topLevel)}
                config={config}
            />
        </Fabric>,
        el)
}

// shim support
if ((BUILD !== "PROD" && typeof runmain !== "undefined" && runmain === true) ||
    EXEC_ENV === "ELECTRON") {
    window.addEventListener("load", () => {
        // @ts-ignore
        run(document.getElementById("container"))
    })
}

You can see that very little code has been touched to adapt to some javascript injected from the main electron process.

My electron start up code is:

const { app, BrowserWindow } = require('electron')
const path = require('path')
const url = require('url')
const fs = require("fs")
const adal = require("adal-node")
const AuthenticationContext = adal.AuthenticationContext

function turnOnLogging() {
    var log = adal.Logging
    log.setLoggingOptions(
        {
            level: log.LOGGING_LEVEL.VERBOSE,
            log: function (level, message, error) {
                console.log(message)
                if (error) {
                    console.log(error)
                }
            }
        })
}
//turnOnLogging()

const argsCmd = process.argv.slice(2);
console.log("ADAL configuration file", argsCmd[0])
const adalConfig = JSON.parse(fs.readFileSync(argsCmd[0]))
global.adalConfig = adalConfig
const authorityHostUrl = adalConfig.authorityHostUrl + "/" + adalConfig.tenant
const context = new AuthenticationContext(authorityHostUrl)
let adalToken = new Promise((res, rej) => {
    context.acquireTokenWithUsernamePassword(adalConfig.acquireTokenResource,
        adalConfig.username,
        adalConfig.password || process.env["DYNAMICS_PASSWORD"],
        adalConfig.applicationId,
        (err, tokenResponse) => {
            if (err) {
                console.log(Error, err)
                global.adalToken = null
                adalToken = null
                rej(err)
            } else {
                console.log("ADAL token response:", tokenResponse)
                adalToken = tokenResponse
                global.adalToken = tokenResponse
                res(adalToken)
            }
        })
})

// refresh with
//context.acquireTokenWithRefreshToken(tokenResponse['refreshToken'], adalConfig.clientId, null, (e, t) => {...})


// Keep a global reference of the window object, if you don't, the window will
// be closed automatically when the JavaScript object is garbage collected.
let win

function createWindow() {
    return adalToken.then(tok => {
        // Create the browser window.
        win = new BrowserWindow({ width: 800, height: 600 })

        // and load the index.html of the app.
        win.loadURL(url.format({
            pathname: path.join(__dirname, "dist", "ttg_", "WebUtilities", "App.electron.html"),
            protocol: 'file:',
            slashes: true
        }))

        // Open the DevTools.
        win.webContents.openDevTools()

        // Emitted when the window is closed.
        win.on('closed', () => {
            // Dereference the window object, usually you would store windows
            // in an array if your app supports multi windows, this is the time
            // when you should delete the corresponding element.
            win = null
        })
    })
}

// This method will be called when Electron has finished
// initialization and is ready to create browser windows.
// Some APIs can only be used after this event occurs.
app.on('ready', createWindow)

// Quit when all windows are closed.
app.on('window-all-closed', () => {
    // On macOS it is common for applications and their menu bar
    // to stay active until the user quits explicitly with Cmd + Q
    if (process.platform !== 'darwin') {
        app.quit()
    }
})

app.on('activate', () => {
    // On macOS it's common to re-create a window in the app when the
    // dock icon is clicked and there are no other windows open.
    if (win === null) {
        createWindow()
    }
})

I won’t show all the gooey webpack config code, but the key is to ensure that your target is set to “electron” so that various node_module/electron* lurking modules are found correctly.

The only key thing about this code, which is almost exactly the code found on the electron website with a small tweak for adal authentication, is that very little needs to be done to do something quickly. Obviously, much more effort is needed to make this more usable e.g. configure some menus.

Note the promise setup on the token. If the authentication takes too long compared to the startup of the embedded web browser, you will have a sequencing issue. We use the promise effect to sequence the startup.

I can run this from my dev directory using npx electron . <path to crm adal .json config file>.

I’ll post the entire project to github at some point.

Wednesday, December 20, 2017

msbuild and merging assemblies.md

msbuild and merging assemblies.md

You probably use ilmerge to merge assemblies for uploadding to dynamics.

The easiest way to enable this in visual studio is by adding the msbuild task ilmerge via nuget. nuget is essentially a fancy “macro” that does something to your project. It often just adds assemblies to your references list or adds a config file to manage some tool that also gets installed into your solution.

Once you do, nuget adds the ilmerge.dll and ilmerge.exe to the “packages” directory in your project. Do not forget to exclude these when you check them into github or another version control system. You can find instructions for adding the ilmerge task here: https://github.com/emerbrito/ILMerge-MSBuild-Task/wiki or through nuget directly https://www.nuget.org/packages/MSBuild.ILMerge.Task/.

An issue arises when you host your project files on a network drive. ilmerge will not work in the msbuild task because it is trying to merge assemblies that are on a network drive and .net does not allow this by default. .net has alot of security hassles in general that make life difficult, but this one is easy to overcome.

To overcome this, you need to realize that msbuild loads assemblies to execute tasks. Hence you need to adjust the permissions that msbuild runs with. msbuild is provided in visual studio’s installation directories. Assuming you have the appropriate permission, find the file: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin (use the proper VS version number for you, mine was 15.0) and adjust the MSBuild.exe.config file:

<?xml version="1.0" encoding="utf-8"?>
  <configuration>
    <runtime>
	    <!-- add this line anywhere in the runtime section -->
		 <loadFromRemoteSources enabled="true"/> 
...

ilmerge will now merge assemblies located at remote resources, such as a network drive, correctly. Maybe everyone knows this already, has another workaround, or maybe no one runs their projects on network shares or its already setup for them. I needed to make a modification for my development environment. I did not find a way to modify this permission globally using .net 4.5.

P.S. Don’t forget you still need to add the dynamics 365 SDK via nuget: https://www.nuget.org/packages/Microsoft.CrmSdk.CoreAssemblies/

Monday, December 18, 2017

dynamics customeraddress loading.md

dynamics customeraddress loading.md

Addresses in dynamics are handled as a special entity with different semantics than most other entities in CRM. You are unable to customize an address by creating new relationships and you are unable to use new form control features as the form is still the old one, much like the connections form.

Addresses (the logical name is customeraddress/customeraddresses) are always allocated along with the primary entities they were designed to serve. For example, 2 address records are created for each contact and account with address numbers 1 and 2. Any additional attributes are numbered 3+ when they are created via an autonumbering scheme. Specific attributes on contact and account are automatically mapped to these two backing addresses. The mapping is two way, so updating the customeraddress record directly also updates the entity. The addresses are only available for listing/managing via an entity’s form navigation menu. The address management screens are not present in the July 2017 release of the UCI client.

One question though, aside from whether you should use the builtin address or not, is how to load them, if you decide to use them. Because the first two addresses are already created, any “insert” operation will leave these blank. If your data source has a primary address identify, you may want to load this into account.address1 (addressnumber = 1). Hence, you need to have a set of addresses, identify the top “1” or “2” addresses, use update, then the rest should be inserted.

Using dynamics-client we can write a small bit of code to do this. dynamics-crm runs under node. You could also do this in any ETL of course.

First we need some helper functions:

 def updateAddress(parentId: String, addressNumber: Int, payload: String) = {
    val q = QuerySpec(
      filter = Option(s"addressnumber eq $addressNumber and _parentid_value eq $parentId"),
      select = Seq("customeraddressid", "addressnumber")
    )
    lift {
      val addresses = unlift(dynclient.getList[CustomerAddress](q.url("customeraddresses")))
      if(addresses.length != 1) throw new Exception(s"Invalid exising customeraddress entity found for ${parentId}-${addressNumber}")
      unlift(dynclient.update("customeraddresses", addresses(0).customeraddressid, payload))
    }
  }

  def insertAddress(payload: String) = dynclient.createReturnId("/customeraddresses", payload)

The update first retrieves the customeraddress via the customeraddress’s parent id then updates the customeraddress record appropriately. Insert is straight forward.

The logic for identifying the primary address needs to come from a SQL command, for example, from the database you are pulling the data from:

 /**
    * Load source side account "location" correctly. First 2 addresses should map to
    * pre-existing addressnumber 1 and 2 and hence should be updates and
    * not inserts. Data should be sorted by crm accountid then sorted by whatever
    * makes the address you want for 1 and 2 appear at the start of the group.
    * 
    * objecttypecode is a string! not a number for this entity: account|contact|...
    */
  val loadAddresses = Action { config =>
    val src: Stream[IO, js.Object] = cats.Applicative[Option].map2(
      config.etl.query orElse config.etl.queryFile.map(Utils.slurp(_)),
      config.etl.connectionFile)(dbStream _)
      .getOrElse(Stream.empty)
    val counter  = new java.util.concurrent.atomic.AtomicInteger(0)
    val xf = xform(config.etl.cliParameters)
    val toPayload = (o: js.Object) => clean(xf(o).asJson)
    val program = src
      .take(config.etl.take.getOrElse(Long.MaxValue))
      .drop(config.etl.drop.getOrElse(0))
      .groupBy(jobj => jobj.asDict[String]("parentid"))
      .map{jobj =>
        val id = jobj._1
        val records = jobj._2
        if(config.etl.verbosity > 0)
          println(s"parentid: ${id}, records: ${records.map(r => Utils.render(r))}")
        counter.getAndAdd(records.length)
        // First two already exist and are updates else inserts
        val updates = records.take(2).zipWithIndex.map{ addr =>
          val parentId = addr._1.asDict[String]("parentid")
          val payload = toPayload(addr._1)
          if(config.etl.verbosity > 2) println(s"Update: parentid=$parentId: ${payload}")
          updateAddress(parentId, 1 + addr._2, payload)
        }
        val inserts = records.drop(2).map{addr =>
          val payload = toPayload(addr)
          if(config.etl.verbosity > 2) println(s"Insert: ${payload}")
          insertAddress(payload)
        }
        (updates ++ inserts).toList.sequence
      }
      .map(Stream.eval(_))

    IO(println("Loading company addresses"))
      .flatMap{_ => program.join(config.common.concurrency).run }
      .flatMap{_ => IO(println(s"# records loaded: ${counter.get()}"))}
  }

The routine method is all you need and took about 30 min to write. There is a standard xf that interprets command line parameters to add, drop or rename attributes. The core logic is in the val program = ... part. Here we just group the input (the input must be sorted on the parentid and address order), take the first two and perform updates while the remaining are inserted. Since the database input probably has some fields that should not be inserted into dynamics, we can specify that those be dropped via the standard xf CLI --drop ‘mysortfield1|mysortfield2’.

That’s it!

Thursday, November 23, 2017

Dynamics CRM, form context without using Xrm.Page, can use with React

If you program in javascript/typescript for forms programming, you know that to access content on the page, you need to use the Xrm.Page object. However, in v9+, Xrm.Page is deprecated. The advice is to obtain the form context off the execution context. The execution context is what is provided when you add a callback handler to an onSave or onChange type event.

If you program with web resources, you know that there is not a supported way to obtain the form context. At best, you typically access it via window.parent.Xrm.Page. That works, but its deprecated. What’s a safer way to obtain the form context?

One way is to setup a simple system of access that relies only on the Form.onLoad handler and publishing the value to a well-know location. A onLoad handler is added in the form editor. Since onLoad is called at a different time then when your web resource may be loaded, you need to setup a simple promise to obtain the form context from any web resource by publishing the form context a well known location that is accesible to all web resources. The best well-known location is on the toplevel window, but for the description below, we add the form context one level up from where the form onLoad handlder is called. This location is also accessible from web resources. Client form scripts and web resources are loaded at the same child level with a common parent iframe.

Here’s how you do it:

  • Setup a function to be called with Form.onLoad.
  • In that function, setup a promise that can be accessed on the onLoad script’s window.parent.
  • In the web resource, access widow.parent. and use a then clause on the promise.

Here’s some code that you would load and attach to the onLoad handler:

/**
 * Captures the form context and stores it in
 * a well known location for other components,
 * especially Web Resources.
 *
 * Since form scripts and web resources live
 * in a hierarchy of iframes, ensure that
 * we embed that knowledge here, once.
 *
 * Note that MS documents are incomplete about
 * the context and its validness after a callback
 * function exits.
 *
 * Usage: arrange to have onLoad called
 * as a form's onload handler.
 */

/** Attachment point for the callback. Object has "Deferred" type. */
function Deferred() {
    return defer(Object.create(Deferred.prototype))
}

/** Add resolve, reject, promise to an object. */
function defer(deferred) {
    deferred.promise = new Promise(function(resolve, reject) {
        deferred.resolve = resolve
        deferred.reject = reject
    })
    return deferred
}

const p = Deferred()

/**
 * Arrange to have this function called
 * with the form's OnLoad event. This is 
 * the only way to guarantee that we obtain
 * a valid form context without going through
 * the deprecated Xrm.Page.
 * 
 * This form assumes that form scripts load
 * into a frame hierarchy that is one below
 * a parent that webresources can also access.
 */
export function onLoad(ctx: any): void {
    p.resolve(ctx.getFormContext())
}

/** 
 * Attach our promise one level up so other frames can find it.
 * To reach the promised land, call FormContextP() to obtain
 * the promise. Use Promise.race (or equivalent) to timeout
 * waiting.
 */
// @ts-ignore
window.parent.FormContextP = p.promise

A web resource would then access window.parent.FormCotextP from its code. There is no other way to pass objects between frame levels in an HTML document. The only thing you have used in the above is that the form script and web resouces are loaded as siblings, however, you can remove that assumption by posting the promise object to the topmost window, if you want.

Assuming you are using react, you could do:

class MyComponent extends React.Component<..,...> {
...
    public componentDidMount(): void {
        const p = (window.parent as any).FormContextP
        if (p)
            p.then(fctx => {
                this.setState({ formContext: (fctx as Xrm.PageContext) })
            })
    }
...
}

I’ve created a highly re-usable EntityForm that captures this and other Dynamics form related information and passes it to the child component. It makes it easy to access key “context” and other information that is needed. Note that in the above, we capture the form context and stick it into state, we could also provide this as “context” to child components by setting up some context methods in the class.

Monday, November 20, 2017

React, Redux, Typescript + Dynamics (Xrm, Crm) Client Programming

I've been assembling notes on react, dynamics and front end programming.

You may find them useful. They are in a state of continuous edit.

gitbook link

Thursday, November 9, 2017

Dynamics, Web API, FetchXml, generating your missing paging cookie

If you use fetchxml with the latest web api, you may be surprised that sometimes you do not get the paging cookie back when your results are > 5000 records. There are alot of articles on the web about using the paging cookie once you do have it.

How do you make sure you get your cookie?

Some people suggest just sticking the paging number into the fetchxml is the answer, but that causes thrash on the server (even if its in the cloud) if you have alot of results to page through.

<fetch page="20" ...>
...
</fetch>

That’s not great as its possible that the server may need to keep running the same query and tossing aside results–maybe a cache will save the day, but maybe not.

The real answer to always generate a paging cookie is contained on MSDN.

You need to add the right odata annotation to have the paging cookie generated.

...code to generate the request
content += 'Prefer: odata.include-annotations="Microsoft.Dynamics.CRM.*"\n'
...

I have not seen this mentioned anywhere so far, so I thought I would write this up. Note that you can use the fully specified annotation Microsoft.Dynamics.CRM.fetchxmlpagingcookie but the .* version picks any other CRM specific annotation that may be out there so I use the .* version vs the specific one. The OData spec has alot of notes on annotations and how to add and remove them. It’s worth a read of course. Don’t forget to add your other annotations e.g. FormattedValues.

There are many API libraries out there that are quite poor in that they do not allow you to easily batch request your fetchxml if your fetchxml string length is too large for the URL variety. Be aware of you what tools you use and their limitations. There is still a URL length limitation in batch requests but it is much larger than the URL limitation. You’ll still need to chunk your fetchxml somehow if you are retrieving, for example, something that requires a large list of values in a condition clause.