Customizing Tilt Brush

Part Two — Custom Tools: Flight

Series So Far:

• Part One — Introduction and Custom Brushes

Welcome back to my series of tutorials on how to customize Tilt Brush, which is now an open-source VR creativity suite. In this part, we’re going to go beyond simply creating new brushes and add completely new functionality to Tilt Brush with a custom tool!

In Tilt Brush, almost everything we do is done with the help of tools. A tool can be something like the Straight Line tool, which applies a modifier to any new brushes created with the Free Paint tool, or it can be something like the Teleport tool, which moves the player. At the end of the day, a tool is simply a way for us to run some code in a nice compartmentalized way.

For now, let’s add in a tool that will let us fly around, rather than using the default teleportation or translation tools. This tutorial will require some C# programming knowledge, so be ye warned!

First, let’s actually create the button in the menu that will eventually be used to access our tool. We’ll place the tool in the advanced tools panel so that we have a bit more room. This panel is a prefab, contained in the Assets/Prefabs/Panels folder, and is named “AdvancedToolsPanel”. If you want to build for Quest, you’ll also need to perform the following operations on the “AdvancedToolsPanel_Mobile” prefab.

Double-click it to open it, and notice that there’s totally space for one more tool in there. We could make the panel bigger easily enough, but for now let’s use that empty space. Move the two bottom buttons to the left to create some space, and then duplicate one of the other buttons to create a new button.

Note that the mirror and straight edge buttons are not tool buttons! The mirror button is a ‘long press button’ and the straight edge button is an ‘option button’. So don’t duplicate either of those. Make sure that your new button has an ‘Option Button’ component on it.

Name your new button object “Button_Fly” and make sure that it’s right underneath the “Button_Straightedge” object in the hierarchy. It’s good to keep things neat!

Take a look at the Tool Button component on your new object. We’ll need to change the Description Text to read “Fly” — we’ll also need a 128x128 black and white image to use as the button texture. Feel free to create one, or use this one I’ve already created:

Place this image in the Assets/Resources/Icons folder and name it “fly.png”. Now you can drag it on to the “Button Texture” property of your new button object.

Finally, we need to change the “Tool” dropdown. But “Fly” isn’t an option! So we’ll need to add it. If you look at the code for ToolButton.cs, you’ll see that m_tool is of type BaseTool.ToolType. So head over to that file (Assets/Scripts/Tools/BaseTool.cs), find the ToolType enum and add a new line to the bottom — FlyTool!

public class BaseTool : MonoBehaviour {
  public enum ToolType {
    SketchSurface,
    Selection,
    ColorPicker,
    BrushPicker,
    BrushAndColorPicker,
    SketchOrigin,
    AutoGif,
    CanvasTool,
    TransformTool,
    StampTool,
    FreePaintTool,
    EraserTool,
    ScreenshotTool,
    DropperTool,
    SaveIconTool,
    ThreeDofViewingTool,
    MultiCamTool,
    TeleportTool,
    RepaintTool,
    RecolorTool,
    RebrushTool,
    SelectionTool,
    PinTool,
    EmptyTool,
    CameraPathTool,
    
    //Custom
    FlyTool,
  }
.....

Once we’ve done that, we can go back to our component and finally select “Fly Tool” as the tool type for our button! Save the prefab and go back to the main scene.

Create a new button in the AdvancedTools panel prefab and set its description, icon and tool type (by modifying the ToolType enum in BaseTool.cs)

Now it’s time to add our tool to the scene. In the main scene (Assets/Scenes/Main.unity), find the object SketchSurface, which is a child of SketchControls. Inside, you’ll see all the tools available, deactivated. Add a new empty child to this object, give it a scale of (0.5, 0.5, 0.5) and call it FlyTool — placing it just above the EmptyTool object.

Any children of this object will be the 3D content that appears on the controller when the tool is activated. I chose to take the line renderer from the ‘bad teleport’ object in the teleport tool and to add a small sphere with the ‘pointer’ material to the end of it. To make things easier on ourselves, let’s place all the visual content as children of an empty ‘DirectionIndicator’ object which is a child of our tool. That will make it easier to turn on/off the indicator objects.

Once you’re finished creating the tool, deactivate it.

Create a new empty tool object with a scale of (0.5, 0.5, 0.5) as a child of SketchControls/SketchSurface, make it look however you like (ensuring that all its visual content is contained within one child), then deactivate it.

Now it’s time to actually create our tool’s functionality. Notice that all the other tools have their own custom components. These components live in the Assets/Scripts/Tools folder. These tools all inherit from the class BaseTool and override a few key methods to do what they need to do.

So let’s create a new script in this folder and call it FlyTool.cs. Before we worry about actually making the user fly, let’s make sure that our tool is integrated and receiving controller input.

Here’s a nice template tool script to get you started. It includes all the code necessary to show/hide the tool at the appropriate times and to ensure the tool stays attached to the controller:

Now we can add this new component to our FlyTool object in the scene, ensure that that tool type is set to FlyTool and press play! You should see your new custom tool in the advanced tool panel, and when you select it, you should see the tool geometry you built appear attached to your hand.

Add the new script to the FlyTool object, select ‘Fly Tool’ for its type, and press play to try it out!

From here, we could add any functionality to the tool that we want, but let’s keep going and add the flight functionality that we’re after. We’d like our flight to be smooth and comfortable, but just to get things up and running, let’s do things in the simplest possible way just so we can see how controller input works in Tilt Brush.

All of this code was extracted by looking at how the TeleportTool works — if you want to get something done ‘the Tilt Brush way’, take a look at the most-similar functionality built into the original app! Modify your UpdateTool function to look like this:

We ask the Tilt Brush input manager to tell us whether the Fly command is currently true or not, and if it is we apply a scene translation equal to the reverse of the tool controller’s forward direction (making it feel like the player is moving in the controller’s forward direction)

Right now there’s an error — there is no “Fly” command in the InputManager SketchCommands! So let’s go and add it. Head to the file Assets/Scripts/InputManager.cs and look for the SketchCommands enum — you simply need to add “Fly” at the bottom!

That removes the error, but right now that command will always be false, since there’s never anything telling it to be true!

So now, scroll down (still in InputManager) until you find the GetCommand function. We need to pass the command through to the Brush (which is the hand that does the drawing.

Update the function to look like this (note lines 37 and 38):

Finally, we can find the Brush.GetCommand function in the file Assets/Scripts/Input/ControllerInfo.cs and update it to look like this (note lines 27 and 28):

Also note that we didn’t need to choose the trigger button here! As you can see, we could easily have chosen a different one!

If you now press play, you should be able to fly around in the direction the tool is pointed while the trigger is pressed!

At this point we’re more-or-less done with the lesson on how to add new tools to Tilt Brush. You should be able to imagine how you could poll different controller buttons, use button combinations to enable different kinds of behaviours, show different visual indications of tool state and inspect existing Tilt Brush tools to investigate ways to use the Tilt Brush API to make changes to your scene. In later tutorials, we’ll investigate how to create tools to let us modify the sketch itself.

You might now like to try adding some more features to your flight tool — acceleration / deceleration would be a nice way to make the flight a little more comfortable, and it would be nice to be able to change the speed using some other controller inputs! This tutorial is just about how to make the tool itself, so I’ll leave it up to you to experiment.

If you’d like to download a complete FlyTool script which includes acceleration, deceleration and an audio effect, you can find it at the gist linked here.

Until next time, happy modding!