.png)
See below a high-level live demo for D365 Guides and Remote Assist standalone.
Now, lets take a look under the hood here. What you see below are the components/building blocks of D365 Guides/Remote assist at its core, how such are connected and also shows how you can connect such to D365FSCM if you are running e.g. the Asset Management/Production Conrol Module here.
Hololens 2:
On the left side, we see Hololens 2. On the Hololens 2 itself we need the Guides/Remote Assist Hololens application installed.
But, what is Microsoft Dynamics 365 Guides?
Guides is one of Microsoft’s mixed-reality application for Microsoft HoloLens that guides operators through the flow of work or a sequence of tasks by providing holographic instructions throughout the process.
Guides itself consists of 3 components:
- The Guides Solution installed on the PowerPlatform/DataVerse
- The Guides PC authoring application installed on your local machine
- The Guides Hololens application installed on the Hololens itself
When initially logging in with your AAD account here, it will also prompt you which DataVerse environment to connect to. (again, its build on top of Dataverse)
Depending on your security rights which are assigned in Powerplatform admin center you can either operate only or both author and operate a guide.
Authoring in that regard means you can position 3D objects based on your anchor – it is not per se creating the guide and the guide cards or steps themselves.
This is done on the Guides PC authoring application - we are talking about that in a second.
Authoring here means more position 3D objects on a specific location based on your anchors.
There are different anchors such as Azure object, holographic or barcode anchor.
1.An Azure Object anchor uses sensing and processing on HoloLens 2 to detect an object and aligns your guide 3D models to a physical object. So, here You don’t have to attach a physical marker or QR code to the machine.
2.A Holographic anchor for example lets you align your guide to a digital 3D hologram that's overlaid on a physical object in the real world manually. SO you align the 3D model manual to your physical environment and confirm the position before starting the guide.
3. But for now we are going with the bar code anchor to keep things simple. (see in the demo)
Dataverse/Powerplatform:
As mentioned above, you will need the Dynamics 365 Guides solution installed in your Power platform environment.
I installed such via Powerplatform admin center. You will need admin rights herefore. (also show how to assign author vs operator rights here)
After installation, you will see such under Apps. Let’s take a look.
The D365 Guides PowerApp looks like this. You can access such via make.powerapps.com.
See the D365 Guides Solution/PowerApp:
Let me tell what you can do there!!!
If you're an admin or an author for Dynamics 365 Guides, you can use the Guides model-driven app to do things that you can't do in the PC app or HoloLens app. For example, you can:
Create workflows in Microsoft Power Apps to automate Dynamics 365 Guides processes
When an author creates a guide using the PC and HoloLens apps, all the data required for that guide (instruction text, images, videos, and 3D parts) is stored in the Microsoft Dataverse
Use PowerAutomate to e.g. to create new category for your 3D objects and a then then add objects to the collection through the Dynamics 365 Guides model-driven app.
That way you could then add objects to the collection through the Dynamics 365 Guides model-driven
Create folders to organize your guides - If you're an admin or an author for Microsoft Dynamics 365 Guides, you can create folders to organize your guides. A folder can contain guides and subfolders. Each folder can be assigned an owner that is inherited by guides or subfolders in that folder. Folders are useful for managing groups of guides with the same permissions, such as guides for a specific team or product.
and much more!
Note: The Guides themselves still need to be created in/via desktop app however– cant be created in PowerApp for now. In fact, if you use the model-driven app to create, update, delete guide step records, you will make Dynamics 365 Guides nonfunctional.
Also it is important to mentione that when an operator uses now the D365 Guides on the HoloLens in Operate mode, each interaction that the operator moves forward or backward is recorded, together with the date and time of the interaction.
This information is stored in two tables in the Microsoft Dataverse:
- Guide Session
- Guide Session Step Visit
Via PowerApps portal you can also create additional PowerApps that you can later reference in the PC authoring app and can actually use when processing the guide on the Hololens in MR.
The Canvas PowerApps I created are e.g. On hand check (shows live inventory from D365FSCM) – You saw such during the demo video...
...as well as the asset warranty checker PowerApp: (fetches live data from D365FSCM Fixed Assets to check the warranty)
PC/Laptop:
On your local machine, The Guides PC application is used to author guides. You can download it from the MSFT Store.First you are prompted to signin with your AAD credentials and then you can select a DataVerse instance to connect to. Now you can create guides themselves, specify the anchor method (QR code anchor vs. Azure anchor for example), add steps, add actions such as PowerApps actions and link actions – I will talk about that later during the demo – and or enhance gudies with 3D objects, guides carts, pictures etc. The properties and guides that you create here are saved in DataVerse. OTB Guides application does not require any coding skill to create mixed-reality guides – its all confguration.After you create the guide on a PC, you will need to log in as an author on the Hololens and you will use the HoloLens app also to connect the holograms to the physical workspace by picking them up and moving them to the correct location.
So, creating guides is a 2 step process.
First creation on the desktop app, then positioning it via athor mode in Hololens app.
We have 2 more applications in that bucket that are actually free downloads that I would recommend to download and check out in context to Hololens 2 – PowerBI desktop and Hololens Desktop streaming Application.
PowerBI desktop I used to connect to the Dataverse Data source. As the guides data is stored in Dataverse we can also use PowerBI to easily create a dashboard for Guides if we would like.
The Hololens Desktop Streaming Application I simply use today to share my view on the Hololens with you. It connects via IPv4 to the Hololens and I can have a live stream that way.
DualWrite:
As you saw Guides is build on top of Powerplatform and Dataverse.
Dual-write is an out-of-box infrastructure that provides near-real-time integration between customer engagement CE apps built in top of DataVerse (such as Dynamics 365 Guides) and finance and operations ERP applications where the data is stored separately in the FO DB. (D365 SCM)
The integration is faciliatetd via entities and DMF – and DW is so to speak the glue between fo and DataVerse applications.
For the MR guide supported asset management inspection list as well as MES MR guide supported manufacturing process scnearios, I simply need to sync the guides entity in DualWrite so that guides created on Dataverse are available on the D365 SCM side for assignment to the BOMS, routes, asset inspection lists, etc.(entity is called mixed reality guides in DualWrite)
Note: You need to log in with a Entra ID user within your tenant (won't be able to see the above when logged in with an Entra ID guest account. Also make sure the popup blocked is not blocking you from accessing the above. This has happened to me a few times in the past.
D365 SCM:
Here we e.g. process production orders and report progress on operation jobs on the production floor execution interface.. (sign options badgeID, number pad, barcode, etc.)
Shop floor workers use the production floor execution interface to register their daily work, such as when they start a job, report feedback about jobs, register indirect activities, and report absence.
Under Production control/Setup/Manufacturing Execution/Design production floor execution and on actionpane/Designtabs, I configure the following.
For my active jobs tab on the MES I selected Guides as available action to the primary actions bar.
The primary action bar is right hand side, secondary is left hand side, main view is middle upper area and detailed view is lower middle area later on the MES.
Then, I exit out to add additional configs such as “enable search” – enable production order filter by entering text or scanning bar code.
We also set up the Mixed Reality Paramaters:
Enter Microsoft Dataverse URL, QR code cell pixcel size, (would start with 15-30 and High as QR code error correction level.Each QR code has one of four levels (of error correction): L, M, Q, H. The level determines the percentage of the total QR code that is allowed to be dirty or damaged without being unable to read. Level L can be dirty/damaged for up to 7%, level M 15%, level Q 25%, level H 30%.
Note: Ensure you have the MRGuidesFeature enabled via Feature Management. For production environments, you must first submit a support ticket to have your tenant added to the flighting group. You also need to enable the license configurations for Asset Management mixed reality and Mixed Reality guide.
Now, I can associate the Mixed Reality guides in D365FSCM to e.g. BOMS, Routes, spcific route operations, inspection lists etc.
When we now use the MES to e.g. record progress on production jobs, I can see a barcode at the MES that I can scan viaa Hololens. It now opens the correct Mixed reality guide in front of me on the Hololens 2 and I get guided through the production process via MR instructions.
See below my thoughts on value add for organizations:
•By implementing safety measures and modelling processes via guides, you can enforce that all required steps are executed in alignment with security standards and in compliance with unified workflows.
•For example, a step to wear a helmet or to ensure the machine is turned off.
2.Another business value could be quality assurance
•You can ensure that maintenance and possible repair work orders are processed/executed in compliance with best practices and in alignment with corporate policies from a quality standpoint – short, minimize errors.
•As you saw, a guide assists a user through a process step by step.
•That way, you can streamline your business processes, making them more efficient and error-free.
• For example, I was guided through a screen replacement step by step today in alignment with my firm’s quality standards and it was hard to actually make an error with the detailed steps provided.
•Sharing knowledge and enhancing capabilities of your workforce drives sustainable success and is important to stay competitive in the market place.
• Via guides, employees can easily be onboarded and trained on a process or operation.
•It provide them all the necessary information easily.
•The nature of interactive guides helps that an employee can undertake a business process or training on their own.
•It doesn’t require any trainer per se.
•It saves time and helps you avoid scheduling troubles.
•Moreover, you can also customize the training guides as per the specific needs of a trainee/training.
•That way, Dynamics 365 Guides is a perfect way for employees to stay updated on changes to the company or their roles.
•For example, I got trained today on a screen replacement and I have never done it before.
•It was easily to comprehend and follow and lastly helped me gaining a new skill.
4.Another point is productivity
•Guides can level up your employees’ productivity, instead of requesting information such as do we have spare parts onhand, does this machine still have warranty, what is the next step to execute in a repair process, a technicans can have this information handy via Guides and PowerApps action while being handsfree and working on the asset itself with 2 hands.
•The handsfree aspect is a big point - We usually use both hands to work efficiently.
•And when it comes to technicians, with their one hand occupied handling a manual, it becomes difficult to perform a task well.
•But with the integration of Dynamics 365 Guides with HoloLens 2, the technicians can easily navigate step-by-step holographic instructions through their HoloLens and use both hands to perform a task.
•For example, I had both hands free while performing the steps of the guides and inquiring about information such is there enough onhand for my spare parts, etc.
5.Lastly, Management could also be a big factor
•Since data of processed guides and maintenance checklists are stored in DataVerse and/or FO, you can easily report on KPIs and metrics via PowerBI.
•Asset Managers could that way easily overlook an overview KPI dashboard and react if needed.
•For example, you could see the PowerBI dashboard that I showed you during the demo.