Development of mixed reality: did you put the speaker in the virtual living room?

We continue the cycle of articles on Windows Mixed Reality and develop from scratch the application with mixed reality on Unity in lab work format (lab work). You can, in parallel with the publication of articles, create such an application. In this part we will look at the support of the motion controller and add the speaker prefab to the virtual living room.

The entire manual will consist of 5 parts. It will be updated every week. Below you can go to the part you need.
')
Part 1: Export a UWP for Windows Mixed Reality application from Unity ;
Part 2: Setting up the environment (VR) ;
Part 3: Controllers ;
Part 4: Spatial sound ;
Part 5: HoloLens and graphics .

Controllers

At this stage of the practical lesson, we will create a prefab speaker and add it to the virtual living room. Then we will add motion controller support to our scene so that the user can move and rotate the speaker.

Adding speaker to the room

1. Go to the Models folder in the Project pane and drag the Speaker3 model to the Hierarchy pane so that this model becomes the root object and not a child of one of the scene objects.
2. Rename the speaker by right-clicking it and choosing the Rename command. Call it Speaker.
3. Select the Speaker object on the scene, then in the Inspector panel, change the following conversion properties:

• for the Scale parameter, set the values ​​(0.02; 0.02; 0.02);
• For the Position parameter, set the values ​​(2.68; 0.14; 2).

Add controller support

Earlier, when we applied Mixed Reality parameters to our scene, we added support for a motion controller.

Let's remember how these parameters affect the operation of motion controllers in our scene.

• The MixedRealityCameraParent element we added has a child object named MotionControllers, and it has a MotionControllerVisualizer script. This script monitors and visualizes the controller model (changes its position, rotates it, and also handles input events).
• The InputManager object we added is waiting for input events from various sources. Among its child objects are objects for the mouse (Mouse), touch input (Touch), gestures (GesturesInput) and Xbox. In the GestureInput object, there is an InteractionInputSource element that waits to receive motion controller events from the InteractionManager. These are motion controller events that will be used to control our speaker.

Speaker control

In order for the user to move and rotate the speaker, you need to add physical properties to it - then we can determine that the look is directed at the speaker.

1. Select the Speaker object in the Hierarchy panel to change its properties and behavior in the Inspector panel.
2. In the Inspector panel, click Add Component, then find and select the Box Collider component.
3. In the added Box Collider component for the Size property, set the values ​​(21, 20, 53) so that it completely surrounds the speaker.

Next, add the Bounding Box (Bounding Box) behavior from the Mixed Reality toolkit:

1. On the Project panel, go to the Holotoolkit → UX → Scripts → BoundingBoxes folder (Holotoolkit → UX → Scripts → Bounding rectangles), select the BoundingBoxRig.cs script and drag this script to the Inspector panel to add it to the components Speaker.

2. Since we do not need to scale the speakers, the Scale Rate parameter can be set to 0.

The BoundingBoxRig component adds a “gizmo” element to the game object (in this case, the dynamics). Using the gizmo, the user can rotate, move and scale the object using motion controllers or HoloLens gestures (touch and manipulation).

3. With the Speaker object selected, click Add Component and in the search window, enter Two Hand Manipulatable, then click the script highlighted in the search results.

4. Set the Manipulation Mode parameter of this component to Rotate to prevent the speakers from scaling.

This scenario allows you to control objects with two hands - with the help of motion controllers or with the help of gestures of pressing and holding in HoloLens.

5. In the Project pane, find the BoundingBoxBasic prefab in the Assets → HoloToolkit → UX → Prefabs → BoundingBoxes folder (Resources → HoloToolkit → UX → Prefabs → Bounding Rectangles) and drag it to the Bounding Box Prefab property (Prefab of the bounding rectangle) Rig in the Speaker object.

6. Also drag the BoundingBoxBasic prefab onto the Bounding Box Prefab property of the Two Hand Manipulateable component.

The BoundingBoxBasic prefab contains visual elements that are drawn during manipulations, so we add it to each component.



7. In the Project pane, find the AppBarCustom prefab in the Assets -> Prefabs folder (Resources -> Prefabs) and drag it to the App Bar Prefab property of the Bounding Box Rig component in the speaker.

We made a lot of changes. Ensure that the Inspector panel of the Speaker game object is as follows:

Running the application and moving the speaker

Save the scene and run it to check the changes.

Try the following:

• select the speaker and move it (the signals of your controller should be monitored);
• click the speaker application toolbar to access the bindings that allow you to rotate the speaker;
• change the size / scale of the speaker (unless you have previously disabled this feature).

→ Original

Authors: Eric Ahlstrom, cbarkerms, Madison Hight