# Needle Engine Documentation

Source: https://docs.needle.tools
---
title: Needle Engine for Blender
editLink: true
---
<br/>
<div class="centered" style="display: flex;
    align-items: center;
    gap: 20px;
    font-size: 2em;
    font-weight: 100;">
    <img src="/logo.png" style="max-height:70px;" title="Needle Logo" alt="Needle Logo"/> +
    <img src="/blender/logo.png" style="max-height:70px;" />
</div>

# Needle Engine for Blender

Needle Engine for Blender allows you to create highly interactive, flexible and lightweight web applications right inside Blender. Use the powerful tools of Blender to visually setup your 3D scenes, animate and design.

## Install the Blender Add-on

<ClientOnly>

Make sure you have installed <a target="_blank" href="https://www.blender.org/download/"><strong>Blender</strong> 4.1 or 4.2</a> and <os-link windows_url="https://nodejs.org/dist/v20.9.0/node-v20.9.0-x64.msi" osx_url="https://nodejs.org/dist/v20.9.0/node-v20.9.0.pkg"><strong>node.js</strong></os-link>. 
</ClientOnly>

<NoDownloadYet>
    <needle-button 
        event_goal="download_blender" 
        event_position="getting_started" 
        large 
        href="https://engine.needle.tools/downloads/blender?utm_source=needle_docs&utm_content=getting_started"
        same_tab
        next_url="/docs/blender/"
        >
        <strong>Download Needle Engine for Blender</strong>
    </needle-button>
</NoDownloadYet>

1. In Blender, go to `Edit > Preferences > Add-ons` and click the drop down arrow to find the `Install from Disk` button. 

2. Select the downloaded zip file (named `needle-blender-plugin-*.zip`) to install it.

3. Search for "Needle" in the Add-ons search bar and make sure `Needle Engine Exporter for Blender` is enabled.


![Settings](/blender/settings.webp)

## Getting Started

Thank you for using Needle Engine for Blender. 

With this add-on you can create highly interactive and optimized WebGL and WebXR experiences inside Blender, that run using Needle Engine and three.js.  

You'll be able to sequence animations, easily lightmap your scenes, add interactivity or create your own scripts written in Typescript or Javascript that run on the web.

<video-embed src="/docs/blender/environment-light.mp4" />
*Matching lighting and environment settings between Blender and Needle Engine. HDRI environment lights are automatically exported, directly from Blender. Once you save, the page is automatically reloaded.*

:::tip Providing Feedback

**Your feedback is invaluable** when it comes to deciding which features and workflows we should prioritize. If you have feedback for us (good or bad), please [let us know in the forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content)!
:::

## Samples for Blender

- [Download Blender Samples](https://engine.needle.tools/downloads/blender/download-samples?utm_source=needle_docs&utm_content=blender)

First create or open a new blend file that you want to be exported to the web.   
Open the Properties window open the scene category. Select a `Project Path` in the Needle Engine panel. Then click `Generate Project`. It will automatically install and start the server - once it has finished your browser should open and the threejs scene will load.
 
![Project panel](/blender/project-panel.webp)

By default your scene will automatically re-exported when you save the blend file.  
If the local server is running (e.g. by clicking `Run Project`) the website will automatically refresh with your changed model.


When your web project already exists and you just want to continue working on the website  
click the blue `Run Project` button to start the local server:  
![Project panel](/blender/project-panel-2.webp)

### Project Panel overview
![Project panel](/blender/project-panel-3.webp)

1) The path to your web project. You can use the little folder button on the right to select a different path.
2) The `Run Project` button shows up when the Project path shows to a valid web project. A web project is valid when it contains a `package.json`
3) `Directory` open the directory of your web project (the `Project Path`)
4) This button re-exports the current scene as a glb to your local web project. This also happens by default when saving your blend file.
5) `Code Editor` tries to open the vscode workspace in your web project
6) If you work with multiple scenes in one blend file, you can configure which scene is your Main scene and should be exported to the web. If any of your components references another scene they will also be exported as separate glb files. When clicking the "Export" button, your Main scene will be the one that's loaded in the browser.
7) Use the `Build: Development` or `Build: Production` buttons when you want to upload your web project to a server. This will bundle your web project and produce the files that you can upload. When clicking `Build: Production` it will also apply optimization to your textures (they will be compressed for the web)
8) Open the documentation



## Blender Settings

### Color Management

By default the blender viewport is set to `Filmic` - with this setting your colors in Blender and in three.js will not match.
To fix this go to the Blender Render category and in the ColorManagement panel select `View Transform`: `Standard`

![Correct color management settings](/blender/settings-color-management.webp)


## Environment Lighting

You can change the environment lighting and skybox using the Viewport shading options.   
Assign a cubemap to use for lighting or the background skybox. You can adjust the strength or blur to modify the appearance to your liking.      

Note: To also see the skybox cubemap in the browser increase the `World Opacity` to 1.

Note: Alternatively you can enable the `Scene World` setting in the Viewport Shading tab to use the environment texture assigned in the blender world settings.

![Environment](/blender/environment.webp)

<video-embed limit_height max_height="300px" src="/docs/blender/environment.mp4" />

Alternatively if you don't want to see the cubemap as a background add a Camera component to your Blender Camera and change `clearFlags: SolidColor` - note that the Camera `backgroundBlurriness` and `backgroundIntensity` settings override the Viewport shading settings.

![Environment Camera](/blender/environment-camera.webp)

### Add your custom HDRI / EXR environment lighting and skybox

<video-embed limit_height src="/docs/blender/custom_hdri.mp4" /> 


## Export

To exclude an object from being exported you can disable the Viewport and the Render display (see image below)

![Exclude from export](/blender/dont-export.webp)


## Animation 🏇

For simple usecases you can use the Animation component for playback of one or multiple animationclips.  
Just select your object, add an Animation component and assign the clip (you can add additional clips to be exported to the clips array.  
By default it will only playback the first clip assigned when `playAutomatically` is enabled. You can trigger the other clips using a simple custom typescript component)  
<video-embed limit_height src="/docs/blender/animation.mp4" />   

### AnimatorController

The animator controller can be created for more complex scenarios. It works as a statemachine which allows you to create multiple animation states in a graph and configure conditions and interpolation settings for transitioning between those. 

<video-embed src="/docs/blender/animatorcontroller-web.mp4" />
*Create and export [animator statemachines](#animatorcontroller) for controlling complex character animations*


#### Creating an AnimatorController

The AnimatorController editor can be opened using the EditorType dropdown in the topleft corner of each panel:

![AnimatorController open window](/blender/animatorcontroller-open.webp)

<video-embed limit_height max_height="188px" src="/docs/blender/animatorcontroller-create.mp4" /> 
*Creating a new animator-controller asset ☝ or select one from your previously created assets*  

##### Graph overview  
![AnimatorController overview](/blender/animatorcontroller-overview.webp)
1) Use `Shift+A` to create a new AnimatorState
2) The `Parameters` node will be created once you add a first node. Select it to setup parameters to be used in transitions (via the Node panel on the right border)
3) This is an AnimatorState. the orange state is the start state (it can be changed using the `Set default state` button in the Node/Properties panel)
4) The Properties for an AnimatorState can be used to setup one or multiple transitions to other states. Use the `Conditions` array to select parameters that must match the condition for doing the transition.

#### Using an AnimatorController

To use an AnimatorController add an Animator component to the root object of your animations and select the AnimatorController asset that you want to use for this object.

![AnimatorController assign to animator](/blender/animatorcontroller-assigning.webp)  

You can set the Animator parameters from typescript or by e.g. using the event of a Button component


### Timeline — NLA Tracks export 🎬

You can export Blender NLA tracks directly to the web.  
Add a PlayableDirector component (via `Add Component`) to a any blender object. Assign the objects in the ``animation tracks`` list in the component for which you want the NLA tracks to be exported.

![](/blender/timeline_setup.webp)  
![](/blender/timeline.webp)  

::: details Code example for interactive timeline playback
Add this script to `src/scripts` (see custom components section) and add it to any object in Blender to make a timeline's time be controlled by scrolling in the browsers

```ts twoslash
import { Behaviour, PlayableDirector, serializable, Mathf } from "@needle-tools/engine";

export class ScrollTimeline extends Behaviour {

    @serializable(PlayableDirector)
    timeline?: PlayableDirector;

    @serializable()
    sensitivity: number = .5;

    @serializable()
    clamp: boolean = false;

    private _targetTime: number = 0;

    awake() {
        this.context.domElement.addEventListener("wheel", this.onWheel);
        if (this.timeline) this.timeline.pause();
    }

    private onWheel = (e: WheelEvent) => {
        if (this.timeline) {
            this._targetTime = this.timeline.time + e.deltaY * 0.01 * this.sensitivity;
            if (this.clamp) this._targetTime = Mathf.clamp(this._targetTime, 0, this.timeline.duration);
        }
    }

    update(): void {
        if (!this.timeline) return;
        const time = Mathf.lerp(this.timeline.time, this._targetTime, this.context.time.deltaTime / .3);
        this.timeline.time = time;
        this.timeline.pause();
        this.timeline.evaluate();
    }
}
```
:::

## Interactivity 😎

You can add or remove components to objects in your hierarchy using the Needle Components panel:

![Component panel](/blender/components-panel.webp)  

![Component panel](/blender/components-panel-select.webp)  
*For example by adding an `OrbitControls` component to the camera object*  
*you get basic camera controls for mobile and desktop devices*
*Adjust settings for each component in their respective panels*

Components can be removed using the X button in the lower right:

![Remove component](/blender/remove-component.webp)

### Custom Components
Custom components can also be easily added by simply writing Typescript classes. They will automatically compile and show up in Blender when saved.  

To create custom components open the workspace via the Needle Project panel and add a `.ts` script file in `src/scripts` inside your web project. Please refer to the [scripting documentation](http://docs.needle.tools/scripting) to learn how to write custom components for Needle Engine.

::: warning Note
Make sure ``@needle-tools/needle-component-compiler`` 2.x is installed in your web project (package.json devDependencies)
::: 

## Lightmapping 💡

Needle includes a lightmapping plugin that makes it very easy to bake beautiful lights to textures and bring them to the web. The plugin will automatically generate lightmap UVs for all models marked to be lightmapped, there is no need to make a manual texture atlas. It also supports lightmapping of multiple instances with their own lightmap data. 
For lightmapping to work, you need at least one light and one object with `Lightmapped` turned on in the `Needle Object` panel.

<video-embed limit_height max_height="800px" src="/docs/blender/lightmapping.mp4" /> 

::: tip
You can download the .blend file from the video [here](https://engine.needle.tools/downloads/blender/lightmaps.blend).
:::
Use the Needle Object panel to enable lightmapping for a mesh object or light: 

![Lightmapping object](/blender/lightmapping-object.webp)

For quick access to lightmap settings and baking options you can use the scene view panel in the `Needle` tab:

![Lightmapping scene panel](/blender/lightmapping-scene-panel.webp)
 
Alternatively you can also use the Lightmapping panel in the `Render Properties` tab:   

![Lightmapping object](/blender/lightmapping-panel.webp)

::: warning Experimental Feature
The lightmapping plugin is experimental. We recommend creating a backup of your .blend file when using it. Please report problems or errors you encounter in [our forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content) 🙏
::: 

## Texture Compression  

The Needle Engine Build Pipeline automatically compresses textures using ECT1S and UASTC (depending on their usage in materials) when making a production build (**requires [toktx](../getting-started/index.md#install-these-tools-for-production-builds) being installed**). But you can override or change the compression type per texture in the Material panel.

You can modify the compression that is being applied per texture. To override the default compression settings go to the `Material` tab and open the `Needle Material Settings`. There you will find a toggle to override the texture settings per texture used in your material. See the [texture compression table](../deployment.md#how-do-i-choose-between-etc1s-uastc-and-webp-compression) for a brief overview over the differences between each compression algorithm.

![Texture Compression options in Blender](/blender/texture-compression.webp)

## Updating

The lightbulb in the Needle Project panel informs you when a new version of the addon is available.  
Simply click the icon to download the new version.    
![Update notification](/blender/updates.webp)

## Reporting an issue

If you run into any problems we're more than happy to help! Please join [our forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content) for fast support.  

Please also check the logs in Blender. You can find logs specific to the Needle Engine Addon via `Help/Needle` in Blender.    

### Integrated Bug Reporter
![Needle Blender Bug Reporter panel](/blender/bugreporter.webp)  
You can also automatically create and upload a bugreport directly from Blender. Uploaded bugreports will solely be used for debugging. They are encrypted on our backend and will be deleted after 30 days.

If needed, in certain cases we're also able to set up custom NDAs for your projects. Please contact us for more information.

:::tip Using the Bug Reporter requires a web project
Make sure you've set up a web project before sending a bug report – it will allow us to understand more about your system and setup and make it easier to reproduce the issue.
:::

# Next Steps

- [Concept: Web Projects](../project-structure.md)
- [Concept: Exporting Assets](../export.md)
- [Concept: Deployment (Share you website)](../deployment.md)
- [Components: Learn about Everywhere Actions](../everywhere-actions.md)
- [Beginner Scripting: Typescript essentials](../getting-started/typescript-essentials.md)
- [Beginner Scripting: How to write custom components](../scripting.md)

---
title: Needle Cloud
description: 'Needle Cloud is an online service. It helps you store, manage, and share 3D assets and apps on the web.
 A variety of file formats are supported, including glTF, USD, FBX, VRM, and more. Spatial web apps made with Needle can be deployed to the cloud directly from the Unity integration, and via command line (CLI).'
---

<br/>
<discountbanner/>


# Needle Cloud

## Overview

Needle Cloud is an online service. It helps you store, manage, and share 3D assets and apps on the web.
A variety of file formats are supported, including glTF, USD, FBX, VRM, and more. Spatial web apps made with Needle can be deployed to the cloud directly from the Unity integration, and via command line (CLI). The Blender integration is coming at a later point; you can use the CLI in the meantime.

Visit [Needle Cloud](https://cloud.needle.tools) to get started.

![Needle Cloud Overview](/cloud/cloud-overview-page.webp)

## Features

1. **Host spatial web apps**  
   Apps made with Needle can be deployed to the cloud directly from our engine integrations. This allows you to give your team and customers public access to apps easily, without having to set up your own server. If needed, you can protect apps with a password.

2. **Manage 3D assets privately and securely**  
   Easily upload and organize your 3D files. Thanks to our fast CDN (content delivery network), your files are stored securely and can be accessed quickly from anywhere in the world.
   Needle Cloud is not a marketplace, and not a social network. It is designed for agencies, studios and creators to manage their assets privately and securely.

3. **Optimize 3D assets from a variety of formats**  
   Automatically compress your files to reduce their size while maintaining visual quality. This makes your files load faster, and saves bandwidth and memory on user's devices.

4. **Sharing and Version Control**  
   Links to your files can be shared with others and used directly in your projects. You can upload new versions of assets and apps. Individual versions can be labelled, which allows for flexible review workflows: for example, you can label a version as `main` or `experimental`. You can also revert labels back to a previous version if needed.

5. **Automation and Pipeline Tools via CLI**  
   The `needle-cloud` CLI (command line interface) makes it easy to automate uploading and optimizing files. This is useful for integrating Needle Cloud into your existing pipeline, or for automating the upload of large numbers of files.

6. **License Management**  
   Licenses for Needle Engine for solo creators and teams are managed through Needle Cloud. This ensures only authorized users can access your files and projects. Contact us for Enterprise and Edu licenses.

## Deploy from Unity

Needle Cloud is integrated into the Unity Editor. This allows you to deploy your apps directly from Unity to Needle Cloud. You can also upload and download assets from Needle Cloud directly in Unity.

1. **Install the Unity integration, if you haven't already.**   
   See [this page](./../unity/) for more info.

2. **Add the `Export Info` component to your scene.**   
   This component is used to configure the export settings for your app.  
   You can use the menu item `GameObject > Needle Engine > Add Export Info` or create a new scene from a Needle template via the menu item `File > New Scene`.

3. **Click on `Upload to Needle Cloud`.**   
   This will build your app, and upload it to Needle Cloud. You can also choose to deploy to a specific team and project. The _upload name_ of the project, visible next to the button, is saved in the scene. Future uploads will use the same upload name, and all uploaded versions will be grouped together on the Needle Cloud website.   
   
   ![Needle Cloud Unity Integration](/cloud/cloud-deploy-v1.webp)

## Deploy from the CLI

Needle Cloud provides a command line interface (CLI) that allows you to manage your assets and deploy your applications efficiently. You can use the CLI to automate tasks and integrate Needle Cloud into your existing workflows.

The CLI is available as an [npm package](https://www.npmjs.com/package/needle-cloud), which means that you need to have Node.js installed on your machine. You can check if you have Node.js installed by running the following command in your terminal:

```bash
node -v
```
If you don't have Node.js installed, you can download it from the [Node.js website](https://nodejs.org/).  

You can install the `needle-cloud` CLI package globally or use it via `npx`. This allows you to run the CLI commands without having to install it globally. 


1. **Use the npx command (recommended)**
   ```bash
   npx needle-cloud deploy '/dist' --team 'My team' --name 'some-project-id'
   ```
2. **Or install needle-cloud globally**  
   A global installation allows use to use the CLI from anywhere on your system. To install the CLI globally, run the following command in your terminal: 
   ```bash
   npm install -g needle-cloud
   ```
   Now, you can use the `needle-cloud` command in your terminal:

   ```bash
   needle-cloud deploy '/dist' --team 'My team' --name 'some-project-id'
   ```

### Automated Deployments
To deploy from **Github Actions** or **Stackblitz** you can provide an access token as `--token <access_token>`. Access tokens can be created on [your team page](https://cloud.needle.tools/team) on Needle Cloud.

```bash
# Deploy to Needle Cloud from e.g. a github action
npx needle-cloud deploy '/path/to/output' --team 'My team' --name 'some name or id' --token '<access_token>'
```

### CLI Help
Use `help` to see all available commandline options and help to individual commands.
```bash
# see all available options
npx needle-cloud help
# get help for a specific command e.g. deploy
npx needle-cloud help deploy
```


## Deployment URLs

When deploying to Needle Cloud, each upload gets a unique URL. You can either share a link to a _specific_ version, or to a _labeled_ version with your team or clients.

-----

In the following example, we have an app that has so far been deployed twice. Each deployment gets a specific URL, also known as a _pinned_ URL since it's pinned to a specific version.
1. [collaborativesandbox-zubcks1qdkhy<strong>-1qdkhy</strong>.needle.run](https://collaborativesandbox-zubcks1qdkhy-1qdkhy.needle.run/)  
   This is the first version that was uploaded.
2. [collaborativesandbox-zubcks1qdkhy<strong>-2e2spt</strong>.needle.run](https://collaborativesandbox-zubcks1qdkhy-2e2spt.needle.run/)  
   This is the second version that was uploaded.

The _latest_ deployment is always available at the following URL. This URL is useful for sharing with your team, as it always points to the most recent version of the app. Another common name for this version is _dev_ or _canary_.
- [collaborativesandbox-zubcks1qdkhy<strong>-latest</strong>.needle.run](https://collaborativesandbox-zubcks1qdkhy-latest.needle.run/)  
  This URL automatically shows the new version when you upload a new version of the app.

The _main_ deployment is useful for sharing with clients, as it always points to the most recent version of the app that you promoted. Other common names for this version are _production_, _stable_, or _live_.
- [collaborativesandbox-zubcks1qdkhy.needle.run](https://collaborativesandbox-zubcks1qdkhy.needle.run/)  
  This URL does not change when you upload a new version. It will only change when you explicitly promote a new version to _main_.

Typically, you upload a new version, review it, and then decide whether you want to promote it to _main_.

-----

The Needle Cloud website shows all deployed versions of the app, including the latest and main versions. Labels can be moved by clicking on <kbd>⋮</kbd> and selecting <kbd>Set main label</kbd> or <kbd>Remove main label</kbd>.  

![Needle Cloud Version List](/cloud/cloud-edit-page.webp)

## Supported 3D Formats

1. **glTF and GLB** <a href="https://cloud.needle.tools/view?file=2oAMeWZ1hWL3C-latest-product" target="_blank">Example</a>   
   The glTF format is the most widely supported format for 3D on the web. It is a lightweight format that can store 3D models, animations, and textures. GLB files are binary versions of glTF files, where all data is stored in a single file.
   glTF supports advanced compression techniques like Draco, KTX2, and Meshopt, which are fully supported by Needle Cloud and Needle Engine.

2. **OpenUSD**   
   USD is a powerful format for 3D data interchange. It is known for its use in the film and VFX industry, and is gaining popularity in the game industry. Needle Cloud supports USDZ and USD files natively through our work on USD-WASM, and also converts USD files to glTF for further processing and optimization.

3. **FBX**  
   FBX has been a popular format for 3D data interchange for many years, but is lacking a number of modern features like PBR materials and extensions. Needle Cloud converts FBX files to glTF for further processing and optimization.  

4. **VRM**  
   VRM is a format for humanoid avatars. It is based on glTF with additional constraints through the use of glTF extensions. Needle Cloud supports VRM files natively, and can optimize them like other glTF files, including complex VRM extensions like phonemes, toon shading and dynamic bones.

5. **OBJ**  
   OBJ is a simple text-based format for 3D models. It supports basic materials through MTL files, animations, and hierarchies of objects. Needle Cloud converts OBJ files to glTF for further processing and optimization. 

:::tip Use glTF or USD when possible
We recommend glTF and USD as the primary formats for 3D data interchange. They are widely supported, have modern features and a good material model.
:::

## Cloud Assets

### Uploading Assets

You can upload your files easily by dragging them into the website or selecting them from your computer.
Non-glTF files are automatically converted to glTF for further processing, but the original files are kept for download and web viewing. 

### Asset Versions

When you visit the Edit Page of an asset, you can see all versions that were uploaded so far by you or your team. You can also tag versions to mark them as "main" or "experimental". "Latest" is the default tag for the most recent version.

### Sharing Links to Assets

You can create links to share specific files or tagged files with your team or clients. Tagged links will automatically update when you move the tag – so you can share a "main" link once and keep updating the file without having to send a new link.

### Using Cloud Assets in Needle Engine

Files stored in Needle Cloud can be brought directly into Needle Engine projects easily. The `Needle Cloud Asset` component takes a link to an asset, and loads it at runtime. This allows you to keep your project size small and load assets on demand that can still be updated in the cloud. 

::: tip Use Progressive Loading where possible
Assets stored on Needle Cloud are automatically optimized for ideal runtime usage using our Progressive Loading technology. For each mesh and texture, multiple level-of-detail versions are generated, and only the parts of the asset that are needed are loaded at runtime. 

This save a lot of bandwidth and memory (typically 90% or more compared to loading the full asset).
:::

### Embedding the Cloud Viewer on Your Website

A fast way to bring 3D to your own website is to **embed** the Needle Cloud viewer. 
To do so, go to an asset's Edit Page and click on <kbd>Embed</kbd>. You can then copy the `iframe` code snippet and paste it into your website.

::: tip Embedding specific versions
You can also embed the viewer with a direct link to the asset, or with a specific tag. This allows you to update the asset on Needle Cloud without having to update the embed code on your website. 
:::

### Embedding in other frameworks

The following embed options are available:
1. **Needle Cloud Viewer**  
   Use the `iframe` code snippet to embed the Needle Cloud viewer on your website.

1. **Needle Engine**  
  Use the provided code snippet to embed Needle Engine on your website as [web component](./../three/).

1. **model-viewer**  
  The [model-viewer](https://modelviewer.dev) project provides a web component for rendering simple, non-interactive 3D models in the browser.

1. **three.js**  
  If you're familiar with three.js, you can use the provided code snippet as a starting point for a three.js app that supports Needle Progressive Loading and efficiently loads files from Needle Cloud.

1. **React-Three-Fiber**  
  If you're using React-Three-Fiber, you can use the provided code snippet as a starting point for a project that supports Needle Progressive Loading and efficiently loads files from Needle Cloud.

1. **Unity**  
  If you're using Unity, you can integrate Needle Cloud assets directly into your projects using the Needle Cloud Asset component for seamless loading and optimization.

### Using Cloud Assets with other engines like Unity or Unreal

There are several ways to use assets stored on Needle Cloud in other engines like Unity or Unreal.
1. **Download and Import**  
   You can download the asset and import it into your project.

2. **Direct Link**   
   You can use the direct link to the asset in your project. This way, you can update the asset on Needle Cloud and it will automatically update in your project. Which link to use depends on the engine and its glTF capabilities:
    - Support for **glTF with Progressive Loading**:   
      Use the `Progressive-World` or `Progressive-Product` link.
      See [npm:@needle-tools/gltf-progressive](https://www.npmjs.com/package/@needle-tools/gltf-progressive) for more information about progressive loading and how to enable it for your engine.

    - Support for **glTF with Draco and KTX2**:
      Use the `Optimized` link.
    - Support for glTF, but **no compression extensions**:  
      Use the `Upload` (for gltf/glb uploads) or `Converted` (for other uploads) link.

3. **Needle Cloud Asset Component**   
   If you are using Needle Engine, you can use the `Needle Cloud Asset` component to load assets at runtime. It will automatically choose the best link for your platform and load the asset with Progressive Loading. This is also supported at runtime in Unity Builds.

## CLI for Assets

The command line interface (CLI) for Needle Cloud allows automating file uploads and compression. The CLI can be used as part of a build step (replacing an asset with an optimized version), or as a standalone tool (for batch processing of files).

See [npm:needle-cloud](https://www.npmjs.com/package/needle-cloud) for more information about the CLI and how to use it.

## FAQ

1. **What is Needle Cloud?**   
   It’s an online service to upload, compress and share 3D assets and scenes.

2. **How do I upload assets to Needle Cloud?**   
   You can upload files by dragging them onto the website, or by uploading them directly from supported integrations. If you have many files, you can use the CLI (command line interface) or the API (application programming interface).

3. **How do I download optimized files from Needle Cloud?**   
   You can download files from the website. Click on `Share` and then `Download`. You can also use the CLI to download files.

4. **Can I share my files with others?**   
   Yes, you can create links to share your files. Links can either be direct download links, or links to the Needle Cloud viewer.

5. **Is there a limit to file sizes?**   
   Upload limits depend on your plan. Check your account details for more info.

6. **Can Needle Cloud files be used with other tools?**   
   Yes, you can use your files in other programs by exporting them as glTF. USD export is coming at a later point.

7. **What happens if I run out of storage space?**   
   You might need to upgrade your plan or delete old files to make room.

<br/>
<div class="centered" style="display: flex;
    align-items: center;
    gap: 20px;
    font-size: 2em;
    font-weight: 100;">
    <img src="/logo.png" style="max-height:70px;" title="Needle Logo" alt="Needle Logo"/> +
    <span style="font-size: 50px;">✨</span> 
</div>

# Integrate with other tools

Needle Engine is designed to be flexible and extensible. It can be integrated with other tools and services to improve your workflow for bringing rich, interactive 3D to the web from any software.

At the core of a custom integration with Needle Engine is the glTF 3D format. This is the most widely supported format for 3D on the web, and the most versatile. This lightweight format can store 3D models, animations, textures, and all kinds of extra data. glTF is extensible, which is exactly why we chose it as the basis for Needle Engine. It allows us to add rich features and interactive capabilities to 3D files, including better rendering, physics, interactions, XR, networking, and more.

As a result of using the standardized glTF format for interchange, creating a basic integration into any software is easy – just export your 3D assets as glTF files and import them into Needle Engine. From there, you can add more features to your integration, to support our scripting extensions. Usually, this is done via a plugin, extension or export hook in your 3D software.

## Structure of a custom integration
The structure of a custom integration looks like this:

1. Export your 3D assets as glTF files. At this point, your integration is likely as simple as clicking the export button in your 3D software.
2. Use the glTF file in a web project using Needle Engine.
   - This can be a web project created with another integration, downloaded from a sample, or a new web project made with `npx needle-create`.
   - Export the glTF file into the `assets` folder. Your web app should automatically refresh whenever you re-export the glTF file.
3. At this point, you have a basic functional integration, and you could already add custom functionality via TypeScript in the web project.
4. The next step is to add a way to create and adjust components in your software. Depending on the software, this can be done through a custom UI, a script, or a plugin.
   - To start, try with a component like `DragControls`. It has a few options, but the defaults are good enough to get started.
   - Then, move onto components that require configuration. A good starting point are our `Everywhere Actions`, because they allow creators to build a wide range of interactive experiences without needing to write any code.
5. Export those components as part of the `NEEDLE_components` glTF extension for each node. Usually, this is done by adding a custom glTF extension or hook to the existing glTF exporter in your software.
6. Integrate with a web project so that UI can be generated for custom components. For Unity and Blender, we call this `Component Compiler` – it automatically creates a UI for the components in your project, and serves as a bridge between TypeScript components and your 3D software.

## Integrate the web project workflow

A fully-flegded integration might also manage more of the web project workflow. All of these operations can be done from the command line, but for ease of use, they can be neatly wrapped in a GUI or a custom menu in your 3D software. This includes:

1. Creating a new project or changing the location of the web project
2. Running the web project from within your 3D software
   - Our [Unity integration](./../unity/) overrides the "Play" button to run the web project.
   - The [Blender integration](./../blender/) has a custom "Play" button that runs the web project.
3. Building the web project to a folder
4. Uploading the built project to Needle Cloud or another platform, and remembering the Project ID and Team ID
   - Our Unity integration additionally shows the last uploads for your team, and allows you to jump to the last deployment of a project.
5. Uploading/downloading individual assets to Needle Cloud or another platform

:::tip Reach out if you're planning to build a custom integration!
Please reach out to us if you are interested in building a custom integration. We are happy to help you with the process, and explain more of the details. For Enterprise customers, we also provide custom integrations as a service.
:::

---
title: Scripting Introduction for Unity Developers
---

Needle Engine provides a tight integration into the Unity Editor. This allows developers and designers alike to work together in a familiar environment and deliver fast, performant and lightweight web-experiences.  

The following guide is mainly aimed at developers with a Unity3D background but it may also be useful for developers with a web or three.js background. It covers topics regarding how things are done in Unity vs in three.js or Needle Engine.

If you are all new to Typescript and Javascript and you want to dive into writing scripts for Needle Engine then we also recommend reading the [Typescript Essentials Guide](./typescript-essentials) for a basic understanding between the differences between C# and Javascript/Typescript.

If you want to code-along you can [open engine.needle.tools/new](https://engine.needle.tools/new) to create a small project that you can edit in the browser ⚡

## The Basics
Needle Engine is a 3d web engine running on-top of [three.js](https://threejs.org/). Three.js is one of the most popular 3D webgl based rendering libraries for the web. Whenever we refer to a `gameObject` in Needle Engine we are *actually* also talking about a three.js `Object3D`, the base type of any object in three.js. Both terms can be used interchangeably. Any `gameObject` *is* a `Object3D`.   

This also means that - if you are already familiar with three.js - you will have no problem at all using Needle Engine. Everything you can do with three.js can be done in Needle Engine as well. If you are already using certain libraries then you will be able to also use them in a Needle Engine based environment.

Note: **Needle Engine's Exporter does _NOT_ compile your existing C# code to Web Assembly**.   
While using Web Assembly _may_ result in better performance at runtime, it comes at a high cost for iteration speed and flexibility in building web experiences. Read more about our [vision](./../vision.md) and [technical overview](./../technical-overview.md). 


:::details How to create a new Unity project with Needle Engine? (Video)
<video-embed src="https://www.youtube.com/watch?v=gZX_sqrne8U" limit_height />  
:::

## Creating a Component
In Unity you create a new component by deriving from `MonoBehaviour`:
```csharp
using UnityEngine;
public class MyComponent : MonoBehaviour { 
}
```

A custom component in Needle Engine on the other hand is written as follows:
```ts twoslash
import { Behaviour } from "@needle-tools/engine"
export class MyComponent extends Behaviour { 
}
```
## Script Fields

### serializable
If you have seen some Needle Engine scripts then you might have noticed that some variables are annotated with `@serializable` above their declaration. This is a Decorator in Typescript and can be used to modify or annotate code. In Needle Engine this is used for example to let the core serialization know which types we expect in our script when it converts from the raw component information stored in the glTF to a Component instance.   
Consider the following example: 
```ts twoslash
import { Behaviour, serializable } from "@needle-tools/engine";
import { Object3D } from "three";

class SomeClass extends Behaviour{
    @serializable(Behaviour)
    myOtherComponent?: Behaviour;
    @serializable(Object3D)
    someOtherObject?: Object3D;
}
```
This tells Needle Engine that `myOtherComponent` should be of type `Behaviour`. It will then automatically assign the correct reference to the field when your scene is loaded. The same is true for `someOtherObject` where we want to deserialize to an `Object3D` reference.  

Note that in some cases the type can be ommitted. This can be done for all [primitive types in Javascript](https://developer.mozilla.org/en-US/docs/Glossary/Primitive). These are `boolean`, `number`, `bigint`, `string`, `null` and `undefined`.
```ts twoslash
import { Behaviour, serializable } from "@needle-tools/engine";
class SomeClass {
    @serializable() // < no type is needed here because the field type is a primitive
    myString?: string;
}
```

### public vs private
Field without any accessor modified like `private`, `public` or `protected` will by default be `public` in javascript  
```ts twoslash
import { Behaviour, serializable } from "@needle-tools/engine";
class SomeClass {
    /// no accessor means it is public:
    myNumber?: number;
    // explicitly making it private:
    private myPrivateNumber?: number;
    protected myProtectedNumber?: number;
}
```
The same is true for methods as well.

## GameObjects and the Scene
To access the current scene from a component you use `this.scene` which is equivalent to `this.context.scene`, this gives you the root three.js scene object.

To traverse the hierarchy from a component you can either iterate over the children of an object   
with a for loop:  
```ts twoslash
for (let i = 0; i < this.gameObject.children; i++) {
    console.log(this.gameObject.children[i]);
}
```
or you can iterate using the `foreach` equivalent:
```ts twoslash
for (const child of this.gameObject.children) {
    console.log(child);
}
```
You can also use three.js specific methods to quickly iterate all objects recursively using the [`traverse`](https://threejs.org/docs/#api/en/core/Object3D.traverse) method:  
```ts twoslash
import { GameObject } from "@needle-tools/engine";
//---cut-before---
this.gameObject.traverse((obj: GameObject) => console.log(obj))
```
or to just traverse visible objects use [`traverseVisible`](https://threejs.org/docs/#api/en/core/Object3D.traverseVisible) instead.

Another option that is quite useful when you just want to iterate objects being renderable you can query all renderer components and iterate over them like so:   
```ts twoslash
import { Renderer } from "@needle-tools/engine";

for(const renderer of this.gameObject.getComponentsInChildren(Renderer))
    console.log(renderer);
```
For more information about getting components see the next section.

## Components
Needle Engine is making heavy use of a Component System that is similar to that of Unity. This means that you can add or remove components to any `Object3D` / `GameObject` in the scene. A component will be registered to the engine when using `addNewComponent(<Object3D>, <ComponentType>)`.   
The event methods that the attached component will then automatically be called by the engine (e.g. `update` or `onBeforeRender`). A full list of event methods can be found in the [scripting documentation](../scripting.md#lifecycle-methods)    

#### Finding Components in the Scene
For getting component you can use the familiar methods similar to Unity. Note that the following uses the `Animator` type as an example but you can as well use any component type that is either built-in or created by you.
| Method name | Desciption |
| --- | --- |
| `this.gameObject.getComponent(Animator)` | Get the `Animator` component on a GameObject/Object3D. It will either return the `Animator` instance if it has an Animator component or `null` if the object has no such componnent. |
| `this.gameObject.getComponentInChildren(Animator)` | Get the first `Animator` component on a GameObject/Object3D or on any of its children
| `this.gameObject.getComponentsInParents(Animator)` | Get all animator components in the parent hierarchy (including the current GameObject/Object3D)

These methods are also available on the static GameObject type. For example ``GameObject.getComponent(this.gameObject, Animator)`` to get the `Animator` component on a passed in GameObject/Object3D. 

To search the whole scene for one or multiple components you can use ``GameObject.findObjectOfType(Animator)`` or `GameObject.findObjectsOfType(Animator)`.

## Renamed Unity Types
Some Unity-specific types are mapped to different type names in our engine. See the following list:  

| Type in Unity | Type in Needle Engine |  |
| -- | -- | -- |
| `UnityEvent` | `EventList` | A UnityEvent will be exported as a `EventList` type (use `serializable(EventList)` to deserialize UnityEvents) |
| `GameObject` | `Object3D` | |
| `Transform` | `Object3D` | In three.js and Needle Engine a GameObject and a Transform are the same (there is no `Transform` component). The only exception to that rule is when referencing a `RectTransform` which is a component in Needle Engine as well. |
| `Color` | `RGBAColor` | The three.js color type doesnt have a alpha property. Because of that all Color types exported from Unity will be exported as `RGBAColor` which is a custom Needle Engine type |

## Transform
Transform data can be accessed on the `GameObject` / `Object3D` directly. Unlike to Unity there is no extra transform component that holds this data.  
- ``this.gameObject.position`` is the vector3 [position](https://threejs.org/docs/?q=obj#api/en/core/Object3D.position) in local space
- ``this.gameObject.worldPosition`` is the vector3 position in world space
- ``this.gameObject.rotation`` is the [euler rotation](https://threejs.org/docs/?q=obj#api/en/core/Object3D.rotation) in local space
- ``this.gameObject.worldRotation`` is the euler rotation in euler angles in world space
- ``this.gameObject.quaternion`` - is the [quaternion rotation](https://threejs.org/docs/?q=obj#api/en/core/Object3D.quaternion) in local space
- ``this.gameObject.worldQuaternion`` is the quaternion rotation in world space
- ``this.gameObject.scale`` - is the vector3 [scale](https://threejs.org/docs/?q=obj#api/en/core/Object3D.scale) in local space
- ``this.gameObject.worldScale`` is the vector3 scale in world space

The major difference here to keep in mind is that `position` in three.js is by default a localspace position whereas in Unity `position` would be worldspace. The next section will explain how to get the worldspace position in three.js.

### WORLD- Position, Rotation, Scale...

In three.js (and thus also in Needle Engine) the `object.position`, `object.rotation`, `object.scale` are all local space coordinates. This is different to Unity where we are used to `position` being worldspace and using `localPosition` to deliberately use the local space position.  

If you want to access the world coordinates in Needle Engine we have utility methods that you can use with your objects. Call `getWorldPosition(yourObject)` to calculate the world position. Similar methods exist for rotation/quaternion and scale. To get access to those methods just import them from Needle Engine like so `import { getWorldPosition } from "@needle.tools/engine"`

Note that these utility methods like `getWorldPosition`, `getWorldRotation`, `getWorldScale` internally have a buffer of Vector3 instances and are meant to be used locally only. This means that you should not cache them in your component, otherwise your cached value will eventually be overriden. But it is safe to call `getWorldPosition` multiple times in your function to make calculations without having to worry to re-use the same instance. If you are not sure what this means you should take a look at the **Primitive Types** section in the [Typescript Essentials Guide](./typescript-essentials.md#primitive-types)

## Time
Use `this.context.time` to get access to time data:  
- `this.context.time.time` is the time since the application started running
- `this.context.time.deltaTime` is the time that has passed since the last frame
- `this.context.time.frameCount` is the number of frames that have passed since the application started
- `this.context.time.realtimeSinceStartup` is the unscaled time since the application has started running  

It is also possible to use `this.context.time.timeScale` to deliberately slow down time for e.g. slow motion effects.


## Raycasting
Use ``this.context.physics.raycast()`` to perform a raycast and get a list of intersections. If you dont pass in any options the raycast is performed from the mouse position (or first touch position) in screenspace using the currently active `mainCamera`. You can also pass in a `RaycastOptions` object that has various settings like `maxDistance`, the camera to be used or the layers to be tested against.

Use ``this.context.physics.raycastFromRay(your_ray)`` to perform a raycast using a [three.js ray](https://threejs.org/docs/#api/en/math/Ray)

Note that the calls above are by default raycasting against visible scene objects. That is different to Unity where you always need colliders to hit objects. The default three.js solution has both pros and cons where one major con is that it can perform quite slow depending on your scene geometry. It may be especially slow when raycasting against skinned meshes. It is therefor recommended to usually set objects with SkinnedMeshRenderers in Unity to the `Ignore Raycast` layer which will then be ignored by default by Needle Engine as well.   

Another option is to use the physics raycast methods which will only return hits with colliders in the scene. 

```ts twoslash
const hit = this.context.physics.engine?.raycast();
```

Here is a editable [example for physics raycast](https://stackblitz.com/edit/needle-engine-physics-raycast-example?file=src%2Fmain.ts,package.json,.gitignore)

## Input
Use ``this.context.input`` to poll input state:

```ts twoslash
import { Behaviour } from "@needle-tools/engine";
export class MyScript extends Behaviour
{
    update(){
        if(this.context.input.getPointerDown(0)){
            console.log("POINTER DOWN")
        }
    }
}
```

You can also subscribe to events in the ``InputEvents`` enum like so:
```ts twoslash
import { Behaviour, InputEvents, NEPointerEvent } from "@needle-tools/engine";

export class MyScript extends Behaviour
{
    onEnable(){
        this.context.input.addEventListener(InputEvents.PointerDown, this.inputPointerDown);
    }
    onDisable() {
        // it is recommended to also unsubscribe from events when your component becomes inactive
        this.context.input.removeEventListener(InputEvents.PointerDown, this.inputPointerDown);
    }

    inputPointerDown = (evt: NEPointerEvent) => { console.log(evt); }
}
```

If you want to handle inputs yourself you can also subscribe to [all events the browser provides](https://developer.mozilla.org/en-US/docs/Web/Events) (there are a ton). For example to subscribe to the browsers click event you can write:
```ts twoslash
window.addEventListener("click", () => { console.log("MOUSE CLICK"); });
```
Note that in this case you have to handle all cases yourself. For example you may need to use different events if your user is visiting your website on desktop vs mobile vs a VR device. These cases are automatically handled by the Needle Engine input events (e.g. `PointerDown` is raised both for mouse down, touch down and in case of VR on controller button down).


## InputSystem Callbacks
Similar to Unity (see [IPointerClickHandler in Unity](https://docs.unity3d.com/Packages/com.unity.ugui@1.0/api/UnityEngine.EventSystems.IPointerClickHandler.html)) you can also register to receive input events on the component itself.

To make this work make sure your object has a ``ObjectRaycaster`` or ``GraphicRaycaster`` component in the parent hierarchy.

```ts twoslash
import { Behaviour, IPointerEventHandler, PointerEventData } from "@needle-tools/engine";

export class ReceiveClickEvent extends Behaviour implements IPointerEventHandler {
    onPointerClick(args: PointerEventData) {
        console.log("Click", args);
    }
}
```

Note: `IPointerEventHandler` subscribes the object to all possible pointer events. The handlers for them are:
- `onPointerDown`
- `onPointerUp`
- `onPointerEnter`
- `onPointerMove`
- `onPointerExit`
- `onPointerClick`

All have a `PointerEventData` argument describing the event.

## Debug.Log
The `Debug.Log()` equivalent in javascript is `console.log()`. You can also use `console.warn()` or `console.error()`.  
```ts twoslash
import { GameObject, Renderer } from "@needle-tools/engine";
const someVariable = 42;
// ---cut-before---

console.log("Hello web");
// You can pass in as many arguments as you want like so:
console.log("Hello", someVariable, GameObject.findObjectOfType(Renderer), this.context);
```

## Gizmos
In Unity you normally have to use special methods to draw Gizmos like `OnDrawGizmos` or `OnDrawGizmosSelected`. In Needle Engine on the other hand such methods dont exist and you are free to draw gizmos from anywhere in your script. Note that it is also your responsibility then to *not* draw them in e.g. your deployed web application (you can just filter them by `if(isDevEnvironment))`).  

Here is an example to draw a red wire sphere for one second for e.g. visualizing a point in worldspace
```ts twoslash
import { Vector3 } from "three";
const hit = { point: new Vector3(0, 0, 0) };
// ---cut-before---
import { Gizmos } from "@needle-tools/engine";
Gizmos.DrawWireSphere(hit.point, 0.05, 0xff0000, 1);
```
Here are some of the available gizmo methods:  
| Method name |  |
| --- | --- |
| `Gizmos.DrawArrow` | |
| `Gizmos.DrawBox` | |
| `Gizmos.DrawBox3` | |
| `Gizmos.DrawDirection` | |
| `Gizmos.DrawLine` | |
| `Gizmos.DrawRay` | |
| `Gizmos.DrawRay` | |
| `Gizmos.DrawSphere` | |
| `Gizmos.DrawWireSphere` | |


## Useful Utility Methods

Import from `@needle-tools/engine` e.g. `import { getParam } from "@needle-tools/engine"`

| Method name | Description |
| --- | --- |
| `getParam()` | Checks if a url parameter exists. Returns true if it exists but has no value (e.g. `?help`), false if it is not found in the url or is set to 0 (e.g. `?help=0`), otherwise it returns the value (e.g. `?message=test`) |
| `isMobileDevice()` | Returns true if the app is accessed from a mobile device |
| `isDevEnvironment()` | Returns true if the current app is running on a local server |
| `isMozillaXR()` | |
| `isiOS` | |
| `isSafari` | |

```ts twoslash
import { isMobileDevice } from "@needle-tools/engine"
if( isMobileDevice() )
```

```ts twoslash
import { getParam } from "@needle-tools/engine"
// returns true 
const myFlag = getParam("some_flag")
console.log(myFlag)
```

## The Web project
In C# you usually work with a solution containing one or many projects. In Unity this solution is managed by Unity for you and when you open a C# script it opens the project and shows you the file.   
You usually install Packages using Unity's built-in package manager to add features provided by either Unity or other developers (either on your team or e.g. via Unity's AssetStore). Unity does a great job of making adding and managing packages easy with their PackageManager and you might never have had to manually edit a file like the `manifest.json` (this is what Unity uses to track which packages are installed) or run a command from the command line to install a package.

In a web environment you use `npm` - the Node Package Manager - to manage dependencies / packages for you. It does basically the same to what Unity's PackageManager does - it installs (downloads) packages from *some* server (you hear it usually called a *registry* in that context) and puts them inside a folder named `node_modules`.     

When working with a web project most of you dependencies are installed from [npmjs.com](https://npmjs.com/). It is the most popular package registry out there for web projects.    

Here is an example of how a package.json might look like: 
```json
{
  "name": "@optional_org/package_name",
  "version": "1.0.0",
  "scripts": {
    "start": "vite --host"
  },
  "dependencies": {
	  "@needle-tools/engine": "^3.5.9-beta",
	  "three": "npm:@needle-tools/three@0.146.8"
	},
  "devDependencies": {
	  "@types/three": "0.146.0",
	  "@vitejs/plugin-basic-ssl": "^1.0.1",
	  "typescript": "^5.0.4",
	  "vite": "^4.3.4",
	  "vite-plugin-compression": "^0.5.1"
	}
}
```

Our default template uses Vite as its bundler and has no frontend framework pre-installed. Needle Engine is unoppionated about which framework to use so you are free to work with whatever framework you like. We have samples for popular frameworks like Vue.js, Svelte, Next.js, React or React Three Fiber.

## Installing packages & dependencies
To install a dependency from npm you can open your web project in a commandline (or terminal) and run `npm i <the/package_name>` (shorthand for `npm install`)  
For example run `npm i @needle-tools/engine` to install [Needle Engine](https://www.npmjs.com/package/@needle-tools/engine). This will then add the package to your `package.json` to the `dependencies` array.  
To install a package as a devDependency only you can run `npm i --save-dev <package_name>`. More about the difference between dependencies and devDependencies below.    

### What's the difference between 'dependencies' and 'devDependencies'
You may have noticed that there are two entries containing *dependency* - `dependencies` and `devDependencies`.   

`dependencies` are **always installed** (or bundled) when either your web project is installed or in cases where you develop a library and your package is installed as a dependency of another project.  

`devDependencies` are **only** installed when developing the project (meaning that when you directly run `install` in the specific directory) and they are otherwise **not** included in your project.


### How do I install another package or dependency and how to use it?
The [Installing](#installing) section taught us that you can install dependencies by running `npm i <package_name>` in your project directory where the `package_name` can be any package that you find on [npm.js](https://npmjs.org).

Let's assume you want to add a tweening library to your project. We will use [`@tweenjs/tween.js`](https://www.npmjs.com/package/@tweenjs/tween.js) for this example. [Here](https://stackblitz.com/edit/needle-engine-tweenjs-example?file=src%2Fmain.ts) is the final project if you want to jump ahead and just see the result.

First run `npm install @tweenjs/tween.js` in the terminal and wait for the installation to finish. This will add a new entry to our package.json:  
```json
"dependencies": {
    "@needle-tools/engine": "^3.5.11-beta",
    "@tweenjs/tween.js": "^20.0.3",
    "three": "npm:@needle-tools/three@0.146.8"
}
```    

Then open one of your script files in which you want to use tweening and import at the top of the file:   
```ts twoslash
import * as TWEEN from '@tweenjs/tween.js';
```
Note that we do here import all types in the library by writing `* as TWEEN`. We could also just import specific types like `import { Tween } from @tweenjs/tween.js`.   

Now we can use it in our script. It is always recommended to refer to the documentation of the library that you want to use. In the case of tween.js they provide a [user guide](https://github.com/tweenjs/tween.js/blob/HEAD/docs/user_guide.md) that we can follow. Usually the Readme page of the package on npm contains information on how to install and use the package.   

To rotate a cube we create a new component type called `TweenRotation`, we then go ahead and create our tween instance for the object rotation, how often it should repeat, which easing to use, the tween we want to perform and then we start it. We then only have to call `update` every frame to update the tween animation. The final script looks like this:
```ts twoslash
import { Behaviour } from "@needle-tools/engine";
import * as TWEEN from '@tweenjs/tween.js';

export class TweenRotation extends Behaviour {

    // save the instance of our tweener
    private _tween?: TWEEN.Tween<any>; 

    start() {
        const rotation = this.gameObject.rotation;
        // create the tween instance
        this._tween = new TWEEN.Tween(rotation);
        // set it to repeat forever
        this._tween.repeat(Infinity);
        // set the easing to use
        this._tween.easing(TWEEN.Easing.Quintic.InOut);
        // set the values to tween
        this._tween.to({ y: Math.PI * 0.5 }, 1000);
        // start it
        this._tween.start();
    }
    
    update() {
        // update the tweening every frame
        // the '?' is a shorthand for checking if _tween has been created
        this._tween?.update();
    }
}
```
Now we only have to add it to any of the objects in our scene to rotate them forever.   
You can see the final script in action [here](https://stackblitz.com/edit/needle-engine-tweenjs-example?file=src%2Fmain.ts).


# Learning more

- [Scripting in Needle Engine](../scripting)
- [Typescript Essentials](./typescript-essentials.md)
- [Component Reference](../component-reference.md)

---
lang: en-US
title: Getting Started & Installation
next: ../project-structure.md
---

<br/>

<discountbanner />


# Downloads

With **Needle Engine**, you can create fully interactive 3D websites using your favorite framework.

Projects created with Needle Engine can be deployed anywhere on the web and get optimized automatically by our state of the art optimization pipeline with automatic LOD support – reducing asset size by up x100 without compromising quality.   

Needle Engine is available as a **package for Unity, add-on for Blender, a ready-to-go Web Component**, or as a npm package for projects without an editor integration.    
Each of these comes with the same components our building blocks and to power to create more – the choice is yours.

## Choose your Workflow

<tool-tiles></tool-tiles>

<!-- | Tool |  |  |
| -- | -- | -- | 
| Node.js **(required)** | 16.x or 18.x <br>[Windows](https://nodejs.org/dist/v18.16.0/node-v18.16.0-x64.msi) <br/> [MacOS](https://nodejs.org/dist/v18.16.0/node-v18.16.0.pkg)   | For running a local development server
| VS Code *(recommended)* | any version<br/>[Windows](https://code.visualstudio.com/sha/download?build=stable&os=win32-x64-user) <br/> [MacOS](https://code.visualstudio.com/sha/download?build=stable&os=darwin-universal) | For code editing (optional)  |
| **Supported Editors** | |
| Unity | 2020.3.16+ <br/>2021.3.9+ <br/>2022.3.0+<br/>[Get Unity Hub](https://unity.com/download) | For setting up your scenes, components, animations... |
| Blender | 3.3<br/>3.4<br/>3.5<br/>3.6<br/>[Get Blender](https://www.blender.org/download/) | For setting up your scenes, components, animations... |
   -->

  
<!-- ### For optimized builds 

| Tool | | |
| -- | -- | -- |
| | | |
| **toktx** | 4.1<br/>[Windows](https://fwd.needle.tools/needle-engine/toktx/win) <br/> [MacOS](https://fwd.needle.tools/needle-engine/toktx/osx) <br/> [Mac OS Apple Silicon](https://fwd.needle.tools/needle-engine/toktx/osx-silicon) <br/> [Other Releases](https://github.com/KhronosGroup/KTX-Software/releases/tag/v4.1.0-rc3)  | For texture compression (recommended) <br/>You can read more about that [here](./deployment.md#production-builds) in our docs -->



<br/>
<br/>



<!--
<img src="/imgs/unity-logo.webp" style="max-height:70px;" />


## Needle Engine for Unity 

*Supported Unity versions: 2021.3 LTS, 2022.3 LTS*

<needle-button event_goal="download_unity" event_position="getting_started" large href="https://engine.needle.tools/downloads/unity?utm_source=needle_docs&utm_content=getting_started"><strong>Download Needle Engine for Unity</strong></needle-button> 

- Drop the downloaded .unitypackage file into a Unity project and confirm that you want to import it.
- Wait a moment for the installation and import to finish. A window may open stating that "A new scoped registry is now available in the Package Manager.". This is our Needle Package registry. You can safely close that window.  
- **Explore Samples** – Select the menu option _Needle Engine > Explore Samples_ to view, open and modify all available [sample scenes](https://engine.needle.tools/samples).  


**See [Needle Engine for Unity](../unity/index.md)** for a full list of features and instructions on getting started.


---


<img src="/blender/logo.png" style="max-height:70px;" />

## Needle Engine for Blender 
*Supported Blender versions: 4.1+*

<needle-button event_goal="download_blender" event_position="getting_started" large href="https://engine.needle.tools/downloads/blender?utm_source=needle_docs&utm_content=getting_started"><strong>Download Needle Engine for Blender</strong></needle-button>  
 
<br/> 

- The Blender add-on is downloaded as a zip file.
- In Blender, go to `File > Settings > Add-ons` and click the `Install` button.
- Then select the downloaded zip to install it.

**See [Needle Engine for Blender](../blender/index.md)** for a full list of features and instructions on getting started.

<br/>
<br/>
<br/>



<br/>
<br/>
<br/>

-->

## Code Editor and Tools

### Install a code editor

Needle Engine makes it easy to build web apps. That often, but not always, includes coding with JavaScript/TypeScript or writing HTML and CSS to describe user interfacces. We recommend [Visual Studio Code](https://code.visualstudio.com) for creating and editing these files. It's a free, open-source code editor that runs on Windows, macOS, and Linux.

<ClientOnly>
<!-- <br/><os-link generic_url="https://engine.needle.tools/downloads/unity">Needle Engine for Unity</os-link> — <os-link generic_url="https://engine.needle.tools/downloads/unity">Needle Engine for Blender</os-link> -->

<os-link windows_url="https://code.visualstudio.com/sha/download?build=stable&os=win32-x64-user" osx_url="https://code.visualstudio.com/sha/download?build=stable&os=darwin-universal">Download Visual Studio Code</os-link>


<br/>
<br/>

### Other useful tools

::: tip
Needle Engine uses the following tools to create your web app, but you don't need to manually install them when using the Unity or Blender integration. We'll guide you through the installation process after you've installed the Needle integration.
:::

<br/>
<os-link windows_url="https://nodejs.org/dist/v22.13.1/node-v22.13.1-x64.msi" osx_url="https://nodejs.org/dist/v22.13.1/node-v22.13.1.pkg">Node.js 20 LTS or 22 LTS.</os-link>
Needle Engine uses Node.js to manage, preview and build the web app that you are creating locally on your computer.     
It is also used for uploading (deploying) your website to the internet.

<br/><os-link windows_url="https://fwd.needle.tools/needle-engine/toktx/win" osx_url="https://fwd.needle.tools/needle-engine/toktx/osx" osx_silicon_url="https://fwd.needle.tools/needle-engine/toktx/osx-silicon">KTX Software – toktx texture tools.</os-link> We use toktx by the Khronos Group to locally optimize and compress your 3D files. Learn more about production builds [in the docs](../deployment.md#production-builds).

<br/>
</ClientOnly>

<!--
## Option 1: Quick Start — Starter Project ⚡
1. **Download or Clone this repository**  
   It's set up with the right packages and settings to get you started right away.  

   _Clone with HTTPS:_ ``https://github.com/needle-tools/needle-engine-support.git``  
   _OR clone with SSH:_ ``git@github.com:needle-tools/needle-engine-support.git``  
   _OR download directly:_ <a href="https://github.com/needle-tools/needle-engine-support/archive/refs/heads/main.zip" target="_blank">Download Repository</a>
   
  
2. **Open the starter project**  
  Open `starter/Needle Engine Starter 2020_3` for a full sandbox project that's ready to run (including a couple of simple example scenes for lightmaps and custom shaders).  
  This is a sandbox builder project! It already comes with multi-player capabilities, and works across mobile, desktop, VR and AR.  

3. **Press Play**  
  Make sure the scene CollaborativeSandbox is open, and press Play! This will automatically do some setup steps and start a local server.  
  Once the setup is complete, a browser window will open, and your project is live.  
  From now on, all changes you do in Unity will be immediately visible in your browser.  

    > **Note**: Your browser might warn you about an untrusted SSL connection. Don't worry, the connection is still encrypted – please click "Advance" if your browser asks you to verify that you're sure you want to visit your server.  

4. **Make it your own**  
  Add assets and components, play around with lighting, add scripts and logic – this is your world now!  
  You can also [publish it on the web for free](#deploy-your-project-to-glitch-) so that others can join you.  
-->



## Next Steps

Now that you've installed Needle Engine, you're ready to dive deeper into project creation, component workflows, scripting, deployment and more. 

- [Getting Started: Unity](../unity/index.md)
- [Getting Started: Blender](../blender/index.md)  
- [Concept: Exporting 3D objects and content](../export.md)
- [Concept: Project Structure](../project-structure.md)
- [Concept: Deploy your website to the web](../deployment.md)
- [Beginner Guide: Typescript Essentials](./typescript-essentials.md) 
- [Beginner Guide: Needle Engine for Unity Developers](./for-unity-developers.md) 
- [Beginner Guide: Scripting Reference](../scripting.md) 
- [Live Examples: Needle Engine Samples](https://engine.needle.tools/samples)

In case you need troubleshooting help, please see the [Questions and Answers – FAQ](../faq.md) section.  
We welcome you to join our [Forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content) and [Discord Community](https://discord.needle.tools).

---
title: Scripting in Needle Engine
description: Differences, similarities and key concepts of Typescript, Javascript and C#.
sidebarDepth: 2
---

The following guide tries to highlight some of the key differences between C#, Javascript and Typescript. This is most useful for developers new to the web ecosystem.

Here are also some useful resources for learning how to write Typescript:  

- [Typescript Tutorial](https://www.typescripttutorial.net/)
- [Learn Typescript](https://www.tutorialsteacher.com/typescript)
- [Typescript Documentation](https://www.typescriptlang.org/docs/)

### Key differences between C#, Javascript or Typescript


**CSharp** or **C#** is a statically typed & compiled language. It means that **before** your code can run (or be executed) it has to be compiled - translated - into IL or CIL, an intermediate language that is a little closer to *machine code*. The important bit to understand here is that your code is analyzed and has to pass certain checks and rules that are **enforced** by the compiler. You will get compiler errors **in Unity** and your application not even start running if you write code that violates any of the rules of the C# language. You will not be able to enter Play-Mode with compiler errors.

**Javascript** on the other hand is interpreted at runtime. That means you can write code that is not valid and cause errors - but you will not see those errors *until your program runs* or tries to **execute** exactly that line that has the error. For example you can write `var points = 100; points += "hello world";` and nobody will complain *until* you run the code in a browser.

**Typescript** is a language designed by Microsoft that **compiles to javascript**  
It adds a lot of features like for example **type-safety**. That means when you write code in Typescript you *can* declare types and hence get errors at *compile-time* when you try to e.g. make invalid assignments or call methods with unexpected types. Read more about types in Javascript and Typescript below. 

### Types — or the lack thereof

**Vanilla Javascript** does (as of today) **not** have any concept of *types*: there is no guarantuee that a variable that you declared as `let points = 100` will still be a *number* later in your application. That means that in Javascript it is perfectly valid code to assign `points = new Vector3(100, 0, 0);` later in your code. Or even `points = null` or `points = myRandomObject` - you get the idea. This is all OK while you write the code **but** it may crash horrible when your code is executed because later you write `points -= 1` and **now** you get errors in the browser when your application is already running.

As mentioned above **Typescript** was created to help fix that problem by adding syntax for defining types.   

It is important to understand that you *basically* still write Javascript when you write Typescript and while it *is* possible to circumvent all type checking and safety checks by e.g. adding ``//@ts-ignore`` above a erroneous line or defining all types as ``any`` this is **definitely not recommneded**. Types are here to help you find errors before they actually happen. You really dont want to deploy your website to your server only to later get reports from users or visitors telling you your app crashed while it was running. 

While *vanilla Javascript* does not offer types you can still add type-annotations to your javascript variables, classes and methods by using **[JSDoc](https://jsdoc.app/)**.

### Variables

In C# you write variables either by using the type or the `var` keyword.   
For example you can either write `int points = 100;`  
or alternatively use `var` and let the compiler figure out the correct type for you: `var points = 100`

In Javascript or Typescript you have two modern options to declaring a variable.  
For a variable that you plan to re-assign use `let`, for example `let points = 100;`  
For a variable that you do not want to be able to re-assign use `const`, for example `const points = 100;`  
> *Be aware of var*   
  You might come across the `var` keyword in javascript as well but it is not recommended to use it and the modern replacement for it is `let`. Learn more about [var vs let](https://stackoverflow.com/a/11444416).

Please note that you *can* still assign values to variables declared with const if they are (for example) a custom type. Consider the following example:  

```ts twoslash
import { Vector3 } from "three";
// ---cut-before---
const myPosition : Vector3 = new Vector3(0, 0, 0);
myPosition.x = 100; // Assigning x is perfectly fine
```
The above is perfectly fine Typescript code because you don't re-assign `myPosition` but only the `x` member of `myPosition`. On the other hand the following example would **not**  be allowed and cause a runtime or typescript error:  
```ts twoslash
// @errors: 2588
import { Vector3 } from "three";
// ---cut-before---
const myPosition : Vector3 = new Vector3(0, 0, 0);
myPosition = new Vector3(100, 0, 0); // ⚠ ASSIGNING TO CONST IS NOT ALLOWED
```

### Using or Importing Types

In Unity you usually add `using` statements at the top of you code to import specific namespaces from Assemblies that are references in your project or - in certain cases - you migth find yourself importing a specific type with a name from a namespace.   
See the following example:
```csharp
using UnityEngine;
// importing just a specific type and giving it a name
using MonoBehaviour = UnityEngine.MonoBehaviour;
```

This is how you do the same in Typescript to import specific types from a package:
```ts twoslash
import { Vector3 } from 'three';
import { Behaviour } from '@needle-tools/engine';
```

You *can* also import all the types from a specific package by giving it a name which you might see here and there:
```ts twoslash
import * as THREE from 'three';
const myVector : THREE.Vector3 = new THREE.Vector3(1, 2, 3);
```

### Primitive Types
*Vector2, Vector3, Vector4...*  
If you have a C# background you might be familiar with the difference between a class and a struct. While a class is a reference type a struct is a custom value type. Meaning it is, depending on the context, allocated on the stack and when being passed to a method by default a copy is created.   
Consider the following example in C#:

```csharp
void MyCallerMethod(){
    var position = new Vector3(0,0,0);
    MyExampleVectorMethod(position);
    UnityEngine.Debug.Log("Position.x is " + position.x); // Here x will be 0
}
void MyExampleVectorMethod(Vector3 position){
    position.x = 42;
}
```

A method is called with a Vector3 named position. Inside the method the passed in vector `position` is modified: x is set to 42. But in C# the original vector that is being passed into this method (see line 2) is **not** changed and x will **still** be 0 (line 4).  

The same is not true for Javascript/Typescript. Here we don't have custom value types, meaning if you come across a Vector in Needle Engine or three.js you will always have a reference type.   
Consider the following example in typescript:  
```ts twoslash
import { Vector3 } from 'three'

function myCallerMethod() : void {
    const position = new Vector3(0,0,0);
    myExampleVectorMethod(position);
    console.log("Position.x is " + position.x); // Here x will be 42
}
function myExampleVectorMethod(position: Vector3) : void {
    position.x = 42;
}
```
Do you see the difference? Because vectors and all custom objects *are* in fact reference types we will have modified the original `position` variable (line 3) and x is now 42.  

This is not only important to understand for methods but also when working with variables.  
In C# the following code will produce two instances of Vector3 and changing one will not affect the other:  
```csharp
var myVector = new Vector3(1,1,1);
var myOtherVector = myVector;
myOtherVector.x = 42;
// will log: 1, 42
UnityEngine.Debug.Log(myVector.x + ", " + myOtherVector.x); 
``` 
If you do the same in Typescript you will **not** create a copy but get a reference to the same `myVector` instance instead:
```ts twoslash
import { Vector3 } from 'three'

const myVector = new Vector3(1,1,1);
const myOtherVector = myVector;
myOtherVector.x = 42;
// will log: 42, 42
console.log(myVector.x, myOtherVector.x); 
```

### Vector Maths and Operators

While in C# you can use operator overloading this is not available in Javascript unfortunately. This means that while you can multiply a Vector3 in C# like this: 

```csharp
var myFirstVector = new Vector3(1,1,1);
var myFactor = 100f;
myFirstVector *= myFactor;
// → myFirstVector is now 100, 100, 100
```

you have to use a method on the Vector3 type to archieve the same result (just with a little more boilerplate code)

```ts twoslash
import { Vector3 } from "three"

const myFirstVector : Vector3 = new Vector3(1, 1, 1)
const myFactor = 100;
myFirstVector.multiplyScalar(myFactor);
// → myFirstVector is now 100, 100, 100
```

### Equality Checks

#### loose vs strict comparison
In C# when you want to check if two variables are the same you can write it as follows:
```csharp
var playerIsNull = myPlayer == null;
```
in Javascript/Typescript there is a difference between `==` and `===` where `===` is more strictly checking for the type:
```ts twoslash
const myPlayer: any = null;
// ---cut-before---
const playerIsNull = myPlayer === null;
const playerIsNullOrUndefined = myPlayer == null;
```
You notice that the second variable `playerIsNullOrUndefined` is using `==` which does a loose equality check in which case `null` and `undefined` will both result in `true`here. You can read more about that [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness)

### Events, Binding and `this`

When you subscribe to an Event in C# you do it like this:
```csharp
// this is how an event is declared
event Action MyEvent;
// you subscribe by adding to (or removing from)
void OnEnable() {
    MyEvent += OnMyEvent;
}
void OnDisable() {
    MyEvent -= OnMyEvent;
}
void OnMyEvent() {}
```
In Typescript and Javascript when you add a method to a list you have to "bind this". That essentially means you create a method where you explictly set `this` to (usually) your current class instance. There are two way to archieve this.   

Please note that we are using the type `EventList` here which is a Needle Engine type to declare events (the EventList will also automatically be converted to a UnityEvent and or a event list in Blender when you use them with our Editor integrations)

The short and **recommended** syntax for doing this is to use [Arrow Functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions).  

```ts twoslash
import { EventList, Behaviour, serializable } from "@needle-tools/engine";

export class MyComponent extends Behaviour {

    @serializable(EventList)
    myEvent!: EventList;

    onEnable() {
        this.myEvent.addEventListener(this.onMyEvent);
    }

    onDisable() {
        this.myEvent.removeEventListener(this.onMyEvent);
    }

    // Declaring the function as an arrow function to automatically bind `this`
    private onMyEvent = () => {
        console.log(this !== undefined, this)
    }
}
```
There is also the more verbose "classical" way to archieve the same thing by manually binding this (and saving the method in a variable to later remove it again from the event list):
```ts twoslash
import { EventList, Behaviour, serializable } from "@needle-tools/engine";

export class MyComponent extends Behaviour {

    @serializable(EventList)
    myEvent?: EventList;
    
    private _onMyEventFn?: Function;

    onEnable() {
        // bind this
        this._onMyEventFn = this.onMyEvent.bind(this);
        // add the bound method to the event
        this.myEvent?.addEventListener(this._onMyEventFn);
    } 
    
    onDisable() {
        this.myEvent?.removeEventListener(this._onMyEventFn);
    }
    
    // Declaring the function as an arrow function to automatically bind `this`
    private onMyEvent = () => { }
}
```

## What's next?

- [Needle Engine Scripting](/scripting.md)

# Get Started
* [Getting Started ⭐](./getting-started/)
    * [Feature Overview](./features-overview.md)
    * [Export](./export.md)
    * [Project Structure](./project-structure.md)
    * [VR and AR](./xr.md)
    * [Scripting](./scripting.md)
    * [HTML](./html.md)
    * [Deployment](./deployment.md)

# Samples
* [Samples and Examples](./samples-and-modules.md)
    * [Tips for Unity Developers](./for-unity-developers.md)
    * [Showcase](./examples.md)
    * [Project Samples, Examples and Modules](./samples-and-modules.md)
    * [FAQ](./faq.md)

# Dive Deeper 🐠
* [Our Vision](./vision.md)
    * [Technical Overview](./technical-overview.md)
    * [Component Reference](./component-reference.md)
    * [Debugging](./debugging.md)

<!-- scripting -->
```mermaid
flowchart LR
  Editor([<b>C# components</b><br/>on GameObjects]) --> gltf[<b>JSON data</b><br/>as glTF Extension] --> Runtime([<b>JavaScript components</b><br/>on Object3D])
  class Editor,gltf,Runtime bg;
``` 



<!-- technical overview -->
```mermaid
flowchart LR
    Editor([Unity Editor]) --> EditorExt([Components + Tools])
    EditorExt -- export data --> glTF([glTF + Extensions])
    glTF --> Bundler([Bundler - vite])
    Runtime([Needle Runtime]) --> Bundler
    Three([Three.js]) --> Bundler
    YourWebsite([Classic web files - HTML, CSS, JS]) --> Bundler
    Bundler -- outputs --> DevPage([web app - dev])
    Bundler -- outputs --> DeploymentPage([web app - deploy])
    glTF -- compressed with --> gltfTransform([glTF-transform]) --> DeploymentPage
    class EditorExt,glTF,Runtime ndl;
    class Editor,Three,Bundler,Page,gltfTransform,DeploymentPage,DevPage,YourWebsite ext;
```

# Documentation Backlog

This section contains pieces of information that are important, but need to be sorted into their correct categories.

## Recommended Unity configuration

- Unity 2020.3.8f1+ or 2022.1+
- Render Pipeline: Universal  
- Color Space: Linear  
- Non-Directional Lightmaps  
- Lightmap Encoding: Normal Quality  

## Supported Unity configurations

- Unity 2020.3+ | Unity 2021.3+ | Unity 2022.1+  
- Render Pipeline: Universal | Built-In<sup>1</sup>  
- Color Space: Linear  

<sup>1</sup>: no custom shader support

## Source Control

Generated Projects can either be added to source control or kept dynamic. Adding them to source control unlocks being able to adjust HTML, CSS, etc very flexible.  
To generate dynamic projects, change their path to `../Library/MyScene`. They will be regenerated if needed.

*Please follow the instructions in the Authentication section if this is your first time accessing packages by needle on this machine.*

# Licensing Setup

> **Note**: This section is deprecated. Needle Engine is currently on Open Beta, and there's no need to authenticate against our registry at this point.

## Authentication  

Make sure you have a Needle Engine and Exporter license, otherwise the following steps will fail (you'll not be able to get authenticated package access).  

*Needs to be setup once per machine.*  

1) Clone this repository and open ``starter/Authenticate`` with Unity 2020.3.x
2) Open [https://packages.needle.tools ⇡](https://packages.needle.tools) in your browser and login (top right corner) with your github account. 
3) Return to [packages.needle.tools ⇡](https://packages.needle.tools) and click the ``i`` icon in the top right corner opening the ``Registry Info`` window.
4) Copy the line containing ``_authToken`` (see the video below)  
   <video src="https://user-images.githubusercontent.com/5083203/166433857-a0c9e29f-9413-4e10-a1a1-2029e3d3ab06.mp4" autoplay></video>
6) Focus Unity - a notification window should open that the information has been added successfully from your clipboard.
7) Click save and close Unity. You should now have access rights to the needle package registry.

Hello world <metalink />

```
meta:

```

### my_test_header :tags webxr, webgl, augmented reality :sample webxr (here is a description)


Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sy
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. 
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looIt uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo

There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage 


### UI Canvas :sample ui canvas (this is a description about UI) :tags hello world

of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo

### How physics works :tags physics :sample Physics Playground :sample another sample

 but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which loo

---
title: Automatic Component Generation
tags:
  - codegen
  - components
  - unity
  - blender
  - editor
  - typescript
  - csharp
---

### Automatically generating Editor components

When working in Unity or Blender then you will notice that when you create a new Needle Engine component in Typescript or Javascript it will automatically generate a Unity C# stub component OR a Blender panel for you.

This is thanks to the magic of the [Needle component compiler](https://www.npmjs.com/package/@needle-tools/needle-component-compiler) that runs behind the scenes in an editor environment and watches changes to your script files. When it notices that you created a new Needle Engine component it will then generate the correct Unity component or Blender panel including public variables or properties that you can then set or link from within the Editor.



### Controlling component generation
You can use the following comments in your typescript code to control C# code generation behavior:  
| Attribute | Result |
| -- | -- |
| `// @generate-component` | Force generation of next class|
| `// @dont-generate-component` | Disable generation of next class, this is useful in cases where you already have an existing C# script in your project |
| `// @serializeField` | Decorate generated field with `[SerializeField]` |
| `// @type UnityEngine.Camera` | Specify generated C# field type |
| `// @nonSerialized` | Skip generating the next field or method |

#### Examples

Force the component compiler to generate a C# AudioClip field named `myAudioClip`
```ts twoslash
import { Behaviour, serializable } from "@needle-tools/engine";

export class MyComponent extends Behaviour {
	//@type UnityEngine.AudioClip
	@serializable()
	myAudioClip?: string;
}

```

Force the component compiler to derive from a specific subclass
```ts twoslash
import { Behaviour } from "@needle-tools/engine";
export class MyCustomBaseClass extends Behaviour { /* ... */ }
// ---cut-before---
//@type MyNamespace.MyCustomBaseClass
export class MyComponent extends MyCustomBaseClass {
}
```


### Component Compiler in Unity
If you want to add scripts inside the ``src/scripts`` folder in your project then you need to have a ``Component Generator`` on the GameObject with your ``ExportInfo`` component.  
Now when adding new components in ``your/threejs/project/src/scripts``it will automatically generate Unity scripts in `Assets/Needle/Components.codegen`.  
If you want to add scripts to any NpmDef file you can just create them - each NpmDef automatically watches script changes and handles component generation, so you don't need any additional component in your scene.

For C# fields to be correctly generated it is currently important that you explictly declare a Typescript type. For example ``myField : number = 5``

You can switch between **Typescript** input and generated **C#** stub components using the tabs below
:::: code-group
::: code-group-item Typescript
```ts twoslash
import { AssetReference, Behaviour, serializable } from "@needle-tools/engine";
import { Object3D } from "three";

export class MyCustomComponent extends Behaviour {
    @serializable()
    myFloatValue: number = 42;

    @serializable(Object3D)
    myOtherObject?: Object3D;

    @serializable(AssetReference)
    prefabs: AssetReference[] = [];

    start() {
        this.sayHello();
    }

    private sayHello() {
        console.log("Hello World", this);
    }
}
```
:::
::: code-group-item Generated C#
```csharp
// NEEDLE_CODEGEN_START
// auto generated code - do not edit directly

#pragma warning disable

namespace Needle.Typescript.GeneratedComponents
{
	public partial class MyCustomComponent : UnityEngine.MonoBehaviour
	{
		public float @myFloatValue = 42f;
		public UnityEngine.Transform @myOtherObject;
		public UnityEngine.Transform[] @prefabs = new UnityEngine.Transform[]{ };
		public void start(){}
		public void update(){}
	}
}

// NEEDLE_CODEGEN_END
```
:::
::: code-group-item Extending Generated C#
```csharp
using UnityEditor;

// you can add code above or below the NEEDLE_CODEGEN_ blocks

// NEEDLE_CODEGEN_START
// auto generated code - do not edit directly

#pragma warning disable

namespace Needle.Typescript.GeneratedComponents
{
	public partial class MyCustomComponent : UnityEngine.MonoBehaviour
	{
		public float @myFloatValue = 42f;
		public UnityEngine.Transform @myOtherObject;
		public UnityEngine.Transform[] @prefabs = new UnityEngine.Transform[]{ };
		public void start(){}
		public void update(){}
	}
}

// NEEDLE_CODEGEN_END

namespace Needle.Typescript.GeneratedComponents
{
    // This is how you extend the generated component (namespace and class name must match!)
	public partial class MyCustomComponent : UnityEngine.MonoBehaviour
	{
		
		public void MyAdditionalMethod()
		{
		}

		private void OnValidate()
		{
			myFloatValue = 42;
		}
	}

    // of course you can also add custom editors
	[CustomEditor(typeof(MyCustomComponent))]
	public class MyCustomComponentEditor : Editor
	{
		public override void OnInspectorGUI()
		{
			EditorGUILayout.HelpBox("This is my sample component", MessageType.None);
			base.OnInspectorGUI();
		}
	}
}

```
:::
::::


### Extending generated components
Component C# classes are generated with the [`partial`](https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/partial-classes-and-methods) flag so that it is easy to extend them with functionality. This is helpful to draw gizmos, add context menus or add additional fields or methods that are not part of a built-in component.  


:::tip Member Casing
Exported members will start with a lowercase letter. For example if your C# member is named ``MyString`` it will be assigned to ``myString``.
:::

---
title: Needle Core Components
---

# Needle Core Components

Here is a overview of some of the components that we provide. Many of them map to components and functionality in Unity, Blender or other integrations.

For a complete list please have a look at our [API docs](https://engine.needle.tools/docs/api/latest).

You can always add your own components or add wrappers for Unity components we haven't provided yet.  

Learn more in the [Scripting](./scripting.md) section of our docs.

## Audio
| Name  | Description |
| ------------- | ------------- |
| `AudioListener` |  |
| `AudioSource` | Use to play audio |

## Animation
| Name  | Description |
| ------------- | ------------- |
| `Animator` with `AnimatorController` | Export with animation state machine, conditions, transitions  |
| `Animation` | Most basic animation component. Only first clip is exported |
| `PlayableDirector` with `TimelineAsset` | Export powerful sequences to control animation, audio, state and more |

## Rendering
| Name  | Description |
| ------------- | ------------- |
| `Camera` |  |
| `Light` | DirectionalLight, PointLight, Spotlight. Note that you can use it to bake light (e.g. Rectangular Light shapes) as well |
| `XRFlag` | Control when objects will be visible. E.g. only enable object when in AR  |
| `DeviceFlag` | Control on which device objects will be visible  |
| `LODGroup` |  |
| `ParticleSystem` | Experimental and currently not fully supported |
| `VideoPlayer` | Playback videos from url or referenced video file (will be copied to output on export). The VideoPlayer also supports streaming from MediaStream objects or `M3U8` livestream URLs |
| `MeshRenderer` | Used to handle rendering of objects including lightmapping and instancing |
| `SkinnedMeshRenderer` | *See MeshRenderer* |
| `SpriteRenderer` | Used to render Sprites and Spriteanimations |
| `Volume` with `PostProcessing` asset | See [table below](#postprocessing) |

### Postprocessing

Postprocessing effects use the [pmndrs postprocessing library](https://www.npmjs.com/package/postprocessing) under the hood. This means you can also easily add your own custom effects and get an automatically optimized postprocessing pass.

- **Unity only**: *Note that Postprocessing effects using a Volume in Unity is only supported with URP*

| Effect Name | |
| --- | --- | 
| Antialiasing | *extra Unity Component* |
| Bloom | *via Volume asset* |
| Chromatic Aberration | *via Volume asset* |
| Color Adjustments / Color Correction | *via Volume asset* | 
| Depth Of Field | *via Volume asset* |
| Vignette | *via Volume asset* |
| ToneMappingEffect | *via Volume asset or separate component* |
| Pixelation | |
| Screenspace Ambient Occlusion N8 | |
| Screenspace Ambient Occlusion | |
| Tilt Shift Effect | |
| SharpeningEffect | |
| *Your custom effect* | [See example on stackblitz](https://stackblitz.com/edit/needle-engine-custom-postprocessing-effect) |

## Networking
| Name  | Description |
| ------------- | ------------- |
| `SyncedRoom` | Main networking component. Put in your scene to enable networking |
| `Networking` | Used to setup backend server for networking. |
| `SyncedTransform` | Automatically network object transformation |
| `SyncedCamera` | Automatically network camera position and view to other users in room. You can define how the camera is being rendered by referencing an object |
| `WebXRSync` | Networks WebXR avatars (AR and VR) |
| `Voip` | Enables voice-chat |
| `Screensharing` | Enables screen-sharing capabilities |

## Interaction
| Name  | Description |
| ------------- | ------------- |
| `EventSystem` | Handles raising pointer events and UI events on objects in the scene |
| `ObjectRaycater` | Required for DragControls and Duplicatable |
| `GraphicsRaycaster` | Same as ObjectRaycaster but for UI elements |
| `DragControls` | Allows objects to be dragged in the scene. Requires raycaster in parent hierarchy, e.g. ObjectRaycaster |
| `Duplicatable` | Can duplicate assigned objects by drag. Requires DragControls |
| `Interactable` | Basic component to mark an object to be interactable. |
| `OrbitControls` | Add to camera to add camera orbit control functionality |
| `SmoothFollow` | Allows to interpolate smoothly to another object's transform |
| `DeleteBox` | Will destroy objects with the `Deletable` component when entering the box |
| `Deletable` | The GameObject this component is attached to will be deleted when it enters or intersects with a `DeleteBox` |
| `DropListener` | Add to receive file drop events for uploading |
| `SpatialTrigger` | Use to raise event if an object enters a specific space or area. You can also use Physics events |
| `SpatialTriggerReceiver` | Use to receive events from SpatialTrigger |

## Physics

Physics is implemented using [Rapier](https://rapier.rs/).  

| Name  | Description |
| ------------- | ------------- |
| `Rigidbody` | Add to make an object react to gravity (or be kinematic and static) |
| `BoxCollider` | A Box collider shape that objects can collide with or raise trigger events when set to `trigger` |
| `SphereCollider` | *See BoxCollider* |
| `CapsuleCollider` | *See BoxCollider* |
| `MeshCollider` | *See BoxCollider* |
| Physics Materials | Physics materials can be used to define e.g. the bouncyness of a collider |

## XR / WebXR  

[Read the XR docs](xr.md)

| Name  | Description |
| ------------- | ------------- |
| `WebXR` | Add to scene for VR, AR and Passthrough support as well as rendering Avatar models |
| [`USDZExporter`](./everywhere-actions.md) | Add to enable USD and Quicklook support
| `XRFlag` | Control when objects are visible, e.g. only in VR or AR or only in ThirdPerson |
| `WebARSessionRoot` | Handles placement and scale of your scene in AR mode |
| `WebARCameraBackground` | Add to access the AR camera image and apply effects or use it for rendering |
| `WebXRImageTracking` | Assign images to be tracked and optionally instantiate an object at the image position |
| `WebXRPlaneTracking` | Create plane meshes or colliders for tracked planes |
| `XRControllerModel` | Can be added to render device controllers or hand models (will be created by default when enabled in the WebXR component) |
| `XRControllerMovement` | Can be added to provide default movement and teleport controls |
| `XRControllerFollow` | Can be added to any object in the scene and configured to follow either left or right hands or controllers |


## Debugging  
| Name  | Description |
| ------------- | ------------- |
| `GridHelper` | Draws a grid |
| `BoxGizmo` | Draws a box |
| `AxesHelper` | Draws XYZ axes |
| | Note: When you're writing custom code you can use the static `Gizmos` methods for drawing debugging lines and shapes | |

## Runtime File Input/Output  
| Name  | Description |
| ------------- | ------------- |
| `GltfExport` | Experimental! Use to export gltf from web runtime. |
| `DropListener` | Receive file drop events for uploading and networking |

## UI

Spatial UI components are mapped from Unity UI (Canvas, not UI Toolkit) to [three-mesh-ui](https://github.com/felixmariotto/three-mesh-ui). 
UI can be animated.   

| Name  | Description |
| ------------- | ------------- |
| `Canvas` | Unity's UI system. Needs to be in World Space mode right now. |
| `Text (Legacy)` | Render Text using Unity's UI Text component. Custom fonts are supported, a font atlas will be automatically generated on export. Use the font settings or the `FontAdditionalCharacters` component to control which characters are included in the atlas.<br/>**Note**: In Unity make sure to use the `Legacy/Text` component (*TextMeshPro* is not supported at the moment) |
| `Button` | Receives click events - use the onClick event to react to it. It can be added too 3D scene objects as well.<br/>**Note**: Make sure to use the `Legacy/Text` component in the Button (or create the Button via the `UI/Legacy/Button` Unity context menu since *TextMeshPro* is not supported at the moment) |
| `Image` | Renders a sprite image |
| `RawImage` | Renders a texture |
| `InputField` | Allows text input |

**Note**: Depending on your project, often a mix of spatial and 2D UI makes sense for cross-platform projects where VR, AR, and screens are supported. Typically, you'd build the 2D parts with HTML for best accessibility, and the 3D parts with geometric UIs that also support depth offsets (e.g. button hover states and the like).  

## Other

| Name  | Description |
| ------------- | ------------- |
| `SceneSwitcher` | Handles loading and unloading of other scenes or prefabs / glTF files. Has features to preload, change scenes via swiping, keyboard events or URL navigation | 


## Editor Only
| Name  | Description |
| --- | --- |
| `ExportInfo` | Main component for managing the web project(s) to e.g. install or start the web app
| `EditorSync` | Add to enable networking material or component value changes to the running three.js app directly from the Unity Editor without having to reload |

---
title: How To Debug
---

## Useful resources for working with glTF

To inspect glTF or glb files online:
- [gltf.report](https://gltf.report/) - three.js based
- [modelviewer.dev/editor](https://modelviewer.dev/editor) - three.js based
- [Khronos glTF Sample Viewer](https://github.khronos.org/glTF-Sample-Viewer-Release/)
- [Babylon Sandbox](https://sandbox.babylonjs.com/)
- [glTF Validator](https://github.khronos.org/glTF-Validator/)

To inspect them locally:
- use the [glTF Shell Extension for Windows](https://apps.microsoft.com/store/detail/gltf-shell-extensions/9NPGVJ9N57MV?hl=en-us&gl=US) to convert between glTF and glb
- use the [glTF Tools VS Code Extension](https://marketplace.visualstudio.com/items?itemName=cesium.gltf-vscode) to see validation errors and in-engine previews locally


## Built-in URL parameters

Debug Flags can be appended as URL query parameters.  
Use ``?help`` to get a list of ALL parameters available.  

Here are some of the most commonly used:

- ``help`` print all available url parameter in the console
- ``console`` opens an on-screen dev console, useful for mobile debugging
- ``printGltf`` logs loaded gltf files to the console
- ``stats`` shows FPS module and logs threejs renderer stats every few seconds
- ``showcolliders`` visualizes physics colliders
- ``gizmos`` enables gizmo rendering (e.g. when using BoxCollider or AxesHelper components)
- and a lot more: please use ``help`` to see them all


## Debug Methods

Needle Engine also has some very powerful and useful debugging methods that are part of the static `Gizmos` class. See the [scripting documentation](./scripting.md#gizmos) for more information.


## Local Testing of release builds
- First, install http-server: `npm install -g http-server`
- make a build (development or production)
- open the *dist* directory with a commandline tool
- run `http-server -g` | *`-g` enables gzip support*  
- optional: if you want to test WebXR, generate a [self-signed SSL certificate](https://stackoverflow.com/a/35231213), then run `http-server -g -S` to enable https (required for WebXR).



## VSCode

You can attach VSCode to the running local server to set breakpoints and debug your code. You can read more about [debugging with VSCode](https://code.visualstudio.com/docs/editor/debugging) here.    

Create a launch.json file at `.vscode/launch.json` in your web project with the following content:  
```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "chrome",
            "request": "launch",
            "name": "Attach Chrome",
            "url": "https://localhost:3000",
            "webRoot": "${workspaceFolder}"
        }
    ]
}
```

If you have changed the port on which your server starts make sure to update the `url` field accordingly.  
You can then start your local server from within VSCode:  
  
![](/debugging/vscode-start-debugging.webp)

## Mobile 

### Android Debugging

For **Android** debugging, you can attach Chrome Dev Tools to your device and see logs right from your PC. You have to switch your device into development mode and connect it via USB.  

See the official chrome documentation [here](https://developer.chrome.com/docs/devtools/remote-debugging/)
- Make sure [Developer Mode](https://developer.android.com/studio/debug/dev-options) is enabled on your phone 
- Connect your phone to your computer via USB
- Open this url in your browser ``chrome://inspect/#devices``
- On your mobile device allow the USB connection to your computer
- On your computer in chrome you should see a list of open tabs after a while (on ``chrome://inspect/#devices``)
- Click ``Inspect`` on the tab you want to debug

### iOS Debugging

For easy iOS debugging add the ``?console`` URL parameter to get a useful on-screen JavaScript console.  

If you have a Mac, you can also attach to Safari (similar to the Android workflow above).   

WebXR usage and debugging on iOS requires using a third-party browser: [Mozilla WebXR Viewer](https://labs.mozilla.org/projects/webxr-viewer/).

### Quest Debugging

Quest is just an Android device - see the [Android Debugging](#android-debugging) section for steps.

---
title: Deployment and Optimization
---

## What does deployment mean?

Deployment is the process of making your application available to the public on a website. Needle Engine ensures that your project is as small and fast as possible by using the latest compression techniques such as **KTX2**, **Draco**, and **Meshopt**.  

## Available Deployment Targets 

- [Needle Cloud](./cloud/#deploy-from-unity) 
  Great for spatial web apps and sharing assets.
- [Glitch](#deploy-to-glitch)
  Great for experimentation and hacking on server-side code.  

- [Netlify](#deploy-to-netlify)
  Great for hosting your own website and custom domain names.
- [itch.io](#deploy-to-itch.io)
  Often used for games.
- [GitHub Pages](#deploy-to-github-pages)
  Free static page hosting.
- [Vercel](#deploy-to-vercel)
  Platform for frontend developers
- [FTP Upload](#deploy-to-ftp)
  Deploy directly to any server with FTP support. Both FTP and SFTP are supported.
- [Build to folder](#build-to-folder)
  When building to a folder, you can upload the files to any web server or other hosting service.
- [Facebook Instant Games](#deploy-to-facebook-instant-games)
  Games platform on Facebook and Facebook Messenger.
  
::: tip Feel something is missing? 
Please let us know in our [forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content)!
:::

## Development Builds

See guides above on how to access the options from within your Editor (e.g. Unity or Blender).  

The main difference to a production build is that it does not perform [ktx2](https://registry.khronos.org/KTX/specs/2.0/ktxspec.v2.html) and [draco](https://google.github.io/draco/) compression (for reduction of file size and loading speed) as well as the option to progressively load high-quality textures.  

We generally recommend making production builds for optimized file size and loading speed (see more information below).

## Production Builds

To make a production build, you need to have [toktx](https://github.com/KhronosGroup/KTX-Software/releases) installed, which provides texture compression using the KTX2 supercompression format. Please go to the [toktx Releases Page](https://github.com/KhronosGroup/KTX-Software/releases) and download and install the latest version (v4.1.0 at the time of writing). You may need to restart Unity after installing it.  
*If you're sure that you have installed toktx and it's part of your PATH but still can't be found, please restart your machine and try build again.*  

:::details Advanced: Custom glTF extensions
If you plan on adding your own custom glTF extensions, building for production requires handling those in ``gltf-transform``. See [@needle-tools/gltf-build-pipeline](https://www.npmjs.com/package/@needle-tools/gltf-build-pipeline) for reference.
:::

### Optimization and Compression Options  

### Texture compression
Production builds will by default compress textures using **KTX2** (either ETC1S or UASTC depending on their usage in the project)   
but you can also select **WebP** compression and select a quality level.  

#### How do I choose between ETC1S, UASTC and WebP compression?

| Format | ETC1S | UASTC | WebP |
| --- | --- | --- | --- |
| **GPU Memory Usage** | Low | Low | High (uncompressed) |
| **File Size** | Low | High | Very low |
| **Quality** | Medium | Very high | Depends on quality setting |
| **Typical usage** | Works for everything, but best for color textures | High-detail data textures: normal maps, roughness, metallic, etc. | Files where ETC1S quality is not sufficient but UASTC is too large |


You have the option to select texture compression and progressive loading options per Texture by using the Needle Texture Importer in Unity or in the Material tab in Blender.
 
:::details Unity: How can I set per-texture compression settings?
![image](/imgs/unity-texture-compression.jpg)  
![image](/imgs/unity-texture-compression-options.jpg)  
:::

:::details Blender: How can I set per-texture compression settings?
Select the material tab. You will see compression options for all textures that are being used by that material.  
![Texture Compression options in Blender](/blender/texture-compression.webp)
:::

:::details Toktx can not be found  
  Windows: Make sure you have added toktx to your system environment variables. You may need to restart your computer after adding it to refresh the environment variables. The default install location is ``C:\Program Files\KTX-Software\bin``    
  ![image](/imgs/ktx-env-variable.webp)
:::

### Mesh compression

By default, a production build will compress meshes using Draco compression. Use the `MeshCompression` component to select between draco and mesh-opt per exported glTF.     
Additionally you can setup mesh simplification to reduce the polycount for production builds in the mesh import settings (Unity). When viewing your application in the browser, you can append `?wireframe` to your URL to preview the meshes.       
  
#### How do I choose between Draco and Meshopt?
| Format | Draco | Meshopt |
| --- | --- | --- |
| **GPU Memory Usage** | Medium | Low |
| **File Size** | Lowest | Low |
| **Animation compression** | No | Yes |


:::details How can I set draco and meshopt compression settings?
Add the MeshCompression component to select which compression should be applied per exported glTF.   

![image](/imgs/unity-mesh-compression-component.jpg)    
- To change compression for the **current scene** just add it anywhere in your root scene.
- To change compression for a **prefab or NestedGltf** add it to a `GltfObject` or the prefab that is referenced / exported by any of your components.
- To change compression for a **referenced scene** just add it to the referenced scene that is exported  
:::

:::details Where to find mesh simplification options to reduce the vertex count when building for production?
Select a Mesh and open the Needle importer options to see available options for the selected mesh:  
![image](/imgs/unity-mesh-simplification.jpg)
:::


### Progressive Textures

You can also add the `Progressive Texture Settings` component anywhere in your scene, to make all textures in your project be progressively loaded. Progressive loading is not applied to lightmaps or skybox textures at this point.   

With progressive loading textures will first be loaded using a lower resolution version. A full quality version will be loaded dynamically when the texture becomes visible. This usually reduces initial loading of your scene significantly.

:::details How can I enable progressive texture loading?
### Progressive textures can be enabled per texture<br/>or for all textures in your project:  
![image](/imgs/unity-texture-compression.jpg)    
### Enable for all textures in the project that don't have any other specific setting:  
![image](/imgs/unity-progressive-textures.jpg)   
:::

### Automatic Mesh LODs (Level of Detail)

Since Needle Engine 3.36 we automatically generate LOD meshes and switch between them at runtime. LODs are loaded on demand and only when needed so so this feature both reduces your loading time as well as performance.

**Key Beneftis**
- Faster initial loading time
- Faster rendering time due to less vertices on screen on average
- Faster raycasting due to the use of LOD meshes

You can either disable LOD generation for your whole project in the `Progressive Loading Settings` component or in the Mesh Importer settings.

![image](/imgs/unity-lods-settings-1.jpg)

![image](/imgs/unity-lods-settings-2.jpg)


## Deployment Options  



### Deploy to Glitch 🎏

[Glitch](https://glitch.com/) provides a fast and free way for everyone to host small and large websites. We're providing an easy way to remix and deploy to a new Glitch page (based on our starter), and also to run a minimalistic networking server on the same Glitch page if needed.  

You can deploy to glitch by adding the `DeployToGlitch` component to your scene and following the instructions.  

Note that free projects hosted on glitch may not exceed ~100 MB. If you need to upload a larger project consider using a different deployment target.

:::details How do I deploy to Glitch from Unity?

1) Add the ``DeployToGlitch`` component to the GameObject that also has the ``ExportInfo`` component.

2) Click the ``Create new Glitch Remix`` button on the component
   ![image](/deployment/deploytoglitch-1.jpg)
3) Glitch will now create a remix of the template. Copy the URL from your browser   
   ![image](https://user-images.githubusercontent.com/5083203/179834901-f28852a9-6b06-4d87-8b5b-0384768c92c1.png)
4) Open Unity again and paste the URL in the ``Project Name`` field of your ``Deploy To Glitch`` component  
  ![image](https://user-images.githubusercontent.com/5083203/179835274-033e5e1d-b70d-4b13-95ad-f1e2f159b14e.png)
5) Wait a few seconds until Unity has received your deployment key from glitch (this key is safely stored in the `.env` file on glitch. Do not share it with others, everyone with this key will be able to upload to your glitch website)  
  ![waiting for the key](/deployment/deploytoglitch-2.jpg)
6) Once the Deploy Key has been received you can click the `Build & Deploy` button to upload to glitch.

::: 

:::details How do I deploy to Glitch from Blender?

![Deploy To Glitch from Blender component](/blender/deploy_to_glitch.webp) 

1) Find the Deploy To Glitch panel in the Scene tab
2) Click the ``Remix on glitch`` button on the component
3) Your browser will open the glitch project template
4) Wait for Glitch to generate a new project
5) Copy paste the project URL in the Blender DeployToGlitch panel as the project name (you can paste the full URL, the panel will extract the necessary information)
6) On Glitch open the ``.env`` file and enter a password in the field ``Variable Value`` next to the **DEPLOY_KEY**  
7) Enter the same password in Blender in the `Key` field
8) Click the `DeployToGlitch` button to build and upload your project to glitch. A browser will open when the upload has finished. Try to refresh the page if it shows black after having opened it.
:::

#### Troubleshooting Glitch

If you click `Create new Glitch Remix` and the browser shows an error like `there was an error starting the editor` you can click **OK**. Then go to [glitch.com](https://glitch.com/) and make sure you are signed in. After that you then try clicking the button again in Unity or Blender.

### Deploy to Netlify  
:::details How do I deploy to Netlify from Unity?
Just add the `DeployToNetlify` component to your scene and follow the instructions. You can create new projects with the click of a button or by deploying to existing projects.  

![Deploy to netlify component](/deployment/deploytonetlify-2.jpg)  

![Deploy to netlify component](/deployment/deploytonetlify.jpg)  
:::

### Deploy to Vercel

1) Create a new project on vercel
2) Add your web project to a github repository
3) Add the repository to your project on vercel


See our [sample project](https://github.com/needle-engine/nextjs-sample) for the project configuration


### Deploy to itch.io  

:::details How do I deploy to itch.io from Unity?
1) Create a new project on [itch.io](https://itch.io/game/new)
2) Set ``Kind of project`` to ``HTML``  
  ![image](https://user-images.githubusercontent.com/5083203/191211856-8a114480-bae7-4bd1-868e-2e955587acd7.png)
3) Add the ``DeployToItch`` component to your scene and click the ``Build`` button  
  ![image](https://user-images.githubusercontent.com/5083203/193812540-1881837e-ed9e-49fc-9658-52e5a914299a.png)

4) Wait for the build to finish, it will open a folder with the final zip when it has finished
5) Upload to final zip to itch.io  
  ![20220920-104629_Create_a_new_project_-_itch io_-_Google_Chrome-needle](https://user-images.githubusercontent.com/5083203/191212661-f626f0cb-bc8e-4738-ad2c-3982aca65f39.png)
6) Select ``This file will be played in the browser``  
  ![image](https://user-images.githubusercontent.com/5083203/191212967-00b687f3-bf56-449e-880c-d8daf8a52247.png)
7) Save your itch page and view the itch project page.  
  It should now load your Needle Engine project 😊
  
#### Optional settings
![image](https://user-images.githubusercontent.com/5083203/191217263-355d9b72-5431-4170-8eca-bfbbb39ae810.png)
:::

:::details Itch.io: failed to find index.html

#### Failed to find index.html
![image](https://user-images.githubusercontent.com/5083203/191213162-2be63e46-2a65-4d41-a713-98c753ccb600.png)  
If you see this error after uploading your project make sure you do not upload a gzipped index.html.
You can disable gzip compression in ``vite.config.js`` in your Needle web project folder. Just remove the line with ``viteCompression({ deleteOriginFile: true })``. The build your project again and upload to itch.  

::: 

### Deploy to FTP

:::details How do I deploy to my FTP server from Unity?
1) Add the ``DeployToFTP`` component¹ on a GameObject in your scene (it is good practice to add it to the same GameObject as ExportInfo - but it is not mandatory) 
2) Assign an FTP server asset and fill out server, username, and password if you have not already ²    
  *This asset contains the access information to your FTP server - you get them when you create a new FTP account at your hosting provider*
3) Click the <kbd>Build & Deploy</kbd> button on the ``DeployToFTP`` component to build your project and uploading it to your FTP account  


![Deploy to FTP component in Unity](/deployment/deploytoftp.jpg)  
*¹ Deploy to FTP component*

![Deploy to FTP server asset](/deployment/deploytoftp2.jpg)  
*² FTP Server asset containing the access information of your FTP user account*

![Deploy to FTP component in Unity with server asset assigned](/deployment/deploytoftp3.jpg)  
*Deploy To FTP component after server asset is assigned. You can directly deploy to a subfolder on your server using the path field* 
:::

:::details How do I deploy to my FTP server manually?

1) Open `File > Build Settings`, select `Needle Engine`, and click on <kbd>Build</kbd>
2) Wait for the build to complete - the resulting `dist` folder will open automatically after all build and compression steps have run.
3) Copy the files from the `dist` folder to your FTP storage.

**That's it!** 😉

![20220830-003602_explorer-needle](https://user-images.githubusercontent.com/2693840/187311461-e6afb2d7-5761-48cf-bacb-1c1733bb768b.png)




> **Note**: If the result doesn't work when uploaded it might be that your web server does not support serving gzipped files. You have two options to fix the problem:   
Option 1: You can try enabling gzip compression on your server using a htaccess file!    
Option 2: You can turn gzip compression off in the build settings at File/Build Window and selecting the Needle Engine platform.  

> **Note**: If you're getting errors during compression, please let us know and report a bug! If your project works locally and only fails when doing production builds, you can get unstuck right away by doing a Development Build. For that, simply toggle `Development Build` on in the Build Settings.

![Unity build window showing Needle Engine platform](/deployment/buildoptions_gzip.jpg)  

:::

#### Enabling gzip using a .htaccess file
To enable gzip compression on your FTP server you can create a file named `.htaccess` in the directory you want to upload to (or a parent directory).  
Insert the following code into your `.htaccess` file and save/upload it to your server:   
```
<IfModule mod_mime.c>
RemoveType .gz
AddEncoding gzip .gz
AddType application/javascript .js.gz
``` 


### Deploy to Github Pages
:::details How do I deploy to Github Pages from Unity?

Add the DeployToGithubPages component to your scene and copy-paste the github repository (or github pages url) that you want to deploy to.  
![Deploy To github pages component](/deployment/deploytogithubpages.jpg) 


<video-embed src="https://www.youtube.com/watch?v=Vyk3cWB6u-c" />


:::

#### Troubleshooting github pages
- **I deployed to github pages but no action is running / the website is not live**    
   - If you deployed for the first time it can take a few minutes until your website becomes available. You can check the **Actions** tab on github (`/actions`) to see the deployment process. 
   - If your website is not live after a few minutes or you don't see any workflow run in the **Actions** tab on github then go to the **Github Pages** settings page (`/settings/pages`) and make sure the **Branch** is set to *gh-pages*



### Deploy to Facebook Instant Games

With Needle Engine you can build to Facebook Instant Games automatically  
No manual adjustments to your web app or game are required.  


:::details How do I deploy to Facebook Instant Games from Unity?
- Add the `Deploy To Facebook Instant Games` component to your scene:
  ![Deploy to facebook instant games component](/deployment/deploytofacebookinstantgames.jpg) 
- Click the `Build For Instant Games` button
- After the build has finished you will get a ZIP file that you can upload to your facebook app.   
- On Facebook add the `Instant Games` module and go to `Instant Games/Web hosting` 
  ![Hosting a facebook instant games](/deployment/deploytofacebookinstantgames-hosting.jpg) 
- You can upload your zip using the `Upload version` button (1). After the upload has finished and the zip has been processed click the `Stage for testing` button to test your app (2, here the blue button) or `Push to production` (the button with the star icon)
  ![Upload the zip to facebook instant games](/deployment/deploytofacebookinstantgames-upload.jpg) 
- That's it - you can then click the `Play` button next to each version to test your game on facebook.

:::


:::details How do I create a app on Facebook (with Instant Games capabilities)

1) [Create a new app](https://developers.facebook.com/apps/creation/) and select `Other`. Then click `Next`
  ![Create facebook instant games app](/deployment/facebookinstantgames-1.jpg) 

2) Select type `Instant Games`
  ![Create facebook instant games app](/deployment/facebookinstantgames-2.jpg) 

3) After creating the app add the `Instant Games` product
  ![Add instant games product](/deployment/facebookinstantgames-3.jpg) 

Here you can find [the official instant games documentation](https://developers.facebook.com/docs/games/build/instant-games) on facebook.  
**Note** that all you have to do is to create an app with instant games capabilities.  
We will take care of everything else and no manual adjustments to your Needle Engine website are required.
:::



## Build To Folder


In Unity open ``File/Build Settings`` and select ``Needle Engine`` for options:

![image](/imgs/unity-build-window-menu.jpg)    

![image](/imgs/unity-build-window.jpg)  

 

To build your web project for uploading to any web server you can click **Build** in the Unity Editor Build Settings Window. You can enable the ``Development Build`` checkbox to omit compression (see below) which requires toktx to be installed on your machine.   

To locally preview your final build you can use the `Preview Build` button at the bottom of the window. This button will first perform a regular build and then start a local server in the directory with the final files so you can see what you get once you upload these files to your webserver.

Nodejs is **only** required during development. The distributed website (using our default vite template) is a static page that doesn't rely on Nodejs and can be put on any regular web server. Nodejs is required if you want to run our minimalistic networking server on the same web server (automatically contained in the Glitch deployment process). 


---

## Cross-Platform Deployment Workflows

It's possible to create regular Unity projects where you can build both to Needle Engine and to regular Unity platforms such as Desktop or even WebGL. Our "component mapping" approach means that no runtime logic is modified inside Unity - if you want you can regularily use Play Mode and build to other target platforms. In some cases this will mean that you have duplicate code (C# code and matching TypeScript logic). The amount of extra work through this depends on your project.  

**Enter Play Mode in Unity**  
In `Project Settings > Needle Engine`, you can turn off `Override Play Mode` and `Override Build settings` to switch between Needle's build process and Unity's build process:  
![image](https://user-images.githubusercontent.com/2693840/187308490-5acb9016-ffff-4113-be62-4de450a42b08.png)



## Needle Engine Commandline Arguments for Unity

Needle Engine for Unity supports various commandline arguments to export single assets (Prefabs or Scenes) or to build a whole web project in batch mode (windowsless).     

The following list gives a table over the available options:  

| | |
| -- | -- |
| `-scene` | path to a scene or a asset to be exported e.g. `Assets/path/to/myObject.prefab` or `Assets/path/to/myScene.unity` |
| `-outputPath <path/to/output.glb>` | set the output path for the build (only valid when building a scene) |
| `-buildProduction` | run a production build | 
| `-buildDevelopment` | run a development build |
| `-debug` | open a console window for debugging |

# Needle Engine on your Website

Needle Engine can be used to create new web apps, and can also be integrated into existing websites. In both cases, you'll want to _upload_ your project's distribution folder to a web hoster to make them accessible to the world. 

There are several ways to integrate Needle Engine with your website. Which one is better depends on a number of factors, like complexity of your project, if you're using custom scripts or only core components, how much control you have over the target website, what the "trust level" is between you and the target website, and so on.

## Try it out

If you want to quickly try out how projects made with Needle will look on your website, just add these two lines anywhere on your page for testing: 

:::: code-group
::: code-group-item Option 1: Embedding Needle
```html
<script type="module" src="https://cdn.jsdelivr.net/npm/@needle-tools/engine/dist/needle-engine.min.js"></script>
<needle-engine src="https://cloud.needle.tools/api/v1/public/873a48a/10801b111/MusicalInstrument.glb"></needle-engine>
```
:::
::: code-group-item Option 2: Using an iframe
```html
<iframe src="https://engine.needle.tools/samples-uploads/musical-instrument/"
    allow="xr; xr-spatial-tracking; fullscreen;" width="100%" height="500px">
</iframe>
```
:::
::: code-group-item Resulting Website
<iframe src="https://engine.needle.tools/samples-uploads/musical-instrument/"
    allow="xr; xr-spatial-tracking; fullscreen;" width="100%" height="500px" style="border:0; outline: 0;">
</iframe>
::::

# Ways to create web apps with Needle

The most common workflows to bring Needle Engine to your website are:
1. [Using the "Deploy to ..." components](#using-the-deploy-to-...-components)
2. [Uploading your web app to a folder](#uploading-your-web-app-to-a-folder)
3. [Embedding a Needle project into an existing website](#embedding-a-needle-project-into-an-existing-website)

## Using the "Deploy to ..." components

Our Needle Engine integrations ship with built-in deployment options. You can deploy your project to Needle Cloud, FTP servers, Glitch, Itch.io, GitHub Pages, and more with just a few clicks. 

See the [Deployment](./deployment.md) section for more information on each of these options.

1. Add the "Deploy to ..." component you want to use to your scene in Unity or Blender.
2. Configure the necessary options and click on "Deploy".
3. That's it! Your project is now live.

:::tip Recommended Workflow
This is the easiest option, and recommended for most workflows – it's very fast! You can iteratively work on your project on your computer, and then upload a new version to the web in seconds.
:::

## Uploading your web app to a folder

If you don't want to use our "Deploy to..." components, or there's no component for your particlar workflow, you can do the same process manually. The resulting web app will be identical to what you see in your local server while working on the project. 

1. Make a production build of your web project. This will create a `dist/` folder with all necessary files, ready for distribution. It contains all necessary files, including the JavaScript bundle, the HTML file, and any other assets like textures, audio, or video files.

2. Upload the content of the `dist/` folder from your Web Project to your web hoster. You can do this via FTP, SFTP, or any other file transfer method your hoster provides. Look at the documentation of your web hoster for details. 

3. That's it! Your web app is now live.


::: tip The folder location influences the URL of your web app.
Depending on your hoster's settings, the folder location and name determine what the URL of your web app is. Here's an example:
- Your domain `https://your-website.com/` points at the folder `/var/www/html` on your webspace.
- You upload your files to `/var/www/html/my-app` so that the `index.html` file is at `/var/www/html/my-app/index.html`.
- The URL of your web app is now `https://your-website.com/my-app/`.
:::

## Embedding a Needle project into an existing website

In some cases, you want a Needle Engine project to be part of an existing web site, for example as a part of a blog post, a product page, or a portfolio. The process is very similar, but instead of uploading the files to the root of your web space, you _embed_ the project into an existing website with a few lines of code.

1. Make a production build of your web project. This will create a `dist/` folder with all necessary files, ready for distribution. It contains all necessary files, including the JavaScript bundle, the HTML file, and any other assets like textures, audio, or video files.

2. Upload the `dist/` folder from your Web Project to your web hoster. 
    ::: tip The folder can be hosted anywhere!
    If you don't have access to your web hoster's file system, or no way to upload files there, you can upload the folder to any other webspace and  use the public URL of that in the next step.
    :::

3. Inside your `dist` folder, you'll find an `index.html` file. We want to copy some lines from this folder, so open the file in a text editor. Typically, it looks like this:  
    ```html
    <head>
        ...
        <script type="module" crossorigin src="./assets/index-732f0764.js"></script>
        ...
    </head>
    <body>
        <needle-engine src="assets/scene.glb"></needle-engine>
    </body>
    ```

    There are two important lines here: 
    - the JavaScript bundle inside `<script>`, 
    - the `<needle-engine>` HTML tag.

4. On the target website, add the `<script...>` and `<needle-engine...>` tags as well. Make sure that the paths point at the location where you have uploaded the files to. 
    ```html
    <script type="module" src="/your-upload-folder/assets/index-732f0764.js"></script>
    <needle-engine src="/your-upload-folder/assets/scene.glb"></needle-engine>
    ```

5. That's it! The scene should now be displayed on your website.

## Embedding a Needle project as iframe

When you have limited access to a website, for example when you're using a CMS like WordPress, you can use an iframe to embed a Needle Engine scene into your website. You may know this workflow from embedding YouTube videos or Sketchfab models.

1. Make a production build of your web project. This will create a `dist/` folder with all necessary files, ready for distribution. 

2. Upload the `dist/` folder from your Web Project to your web hoster.
    ::: tip The folder can be hosted anywhere!
    If you don't have access to your web hoster's file system, or no way to upload files there, you can upload the folder to any other webspace and  use the public URL of that in the next step.
    :::

3. Add an iframe to your website, pointing to the `index.html` file in the `dist/` folder. 
    ```html
    <iframe
        src="https://your-website.com/needle-files/dist/index.html" 
        allow="xr; xr-spatial-tracking; fullscreen;">
    </iframe>
    ```


    ::: tip Permissions inside iframes
    The list inside `allow=` depends on the features your web app uses. For example, XR applications require `xr` and `xr-spatial-tracking` to work inside iframes. 

    There may be additional features needed, for example `camera; microphone; display-capture; geolocation`. See [the full list of iframe Permissions Policy directives on MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy#directives).
    :::

4. That's it! The scene should now be displayed on your website.

## Embedding scenes that use no custom scripts

When your project uses only core components and no custom scripts, you can directly use Needle Engine from a CDN (content-delivery network).

1. Add the following snippet to your website, for example as "HTML Block" in your CMS:
    ```html
    <script type="module" src="https://cdn.jsdelivr.net/npm/@needle-tools/engine/dist/needle-engine.min.js"></script>
    <needle-engine src="https://cloud.needle.tools/api/v1/public/873a48a/10801b111/MusicalInstrument.glb" background-blurriness="0.8"></needle-engine>
    ```
2. Upload the `assets/` folder from your Web Project to your web hoster. Depending on your project settings, this folder contains one or more `.glb` files and any number of other files like audio, video, skybox and more. 

3. Change the `src=` attribute of the `needle-engine` tag to the URL of the `.glb` file you want to display. Typically, this will be some path like `https://your-website.com/assets/MyScene.glb`. 

4. That's it! The scene should now be displayed on your website.

## Embedding a Needle Cloud web app as iframe 

If you deployed your project to Needle Cloud, you can easily display it on your own website with an iframe.  

::: warning <b>Under construction.</b> This section is not yet complete.
:::

# Common Workflows

## Creating a web app for a client's website

1. **Understand what type of app you're building**, and if and how it connects to an existing website.  
   Often, you're building a standalone app that is accessible from a link on the client's domain.  
   But there might also be other server-side and client-side components involved. 

2. **Understand which URL the web app should be accessible from.**  
  This could either be 

    - A page on **[Needle Cloud](./cloud/)**  
      `collaborativesandbox-zubcks1qdkhy.needle.run`  

    - A **Subpage** on the client's website   
      `my-page.com/app`
  
    - A new **Subdomain**  
      `app.my-page.com`
    - A new or existing **Domain**  
      `my-app.com` 

    ::: tip There's no "good" or "bad" here.
    A typical approach is to start on [Needle Cloud](./cloud/) for initial prototypes and during development, and move to the client's webspace and domain for the final version.  

    The choice mostly depends on the client's requirements regarding branding, SEO, and technical setup. Often, you'll have to discuss this with the client's IT department or webmaster. 
    :::

1. **Understand how the web app will be deployed and maintained.**  
    - Will you have access to a folder on the client's web server so you can upload the latest version, or do they want to manage the deployment themselves?
      ::: tip A simple approach: FTP access
      Often, you can ask for FTP or SFTP access to a folder on the client's web server. You'll get a URL, username, and password, and then you can upload your files to that folder. We provide a "Deploy to FTP" component that makes this particularly easy. The client's IT department will set up which URL the folder is accessible from.
        :::

    - Is there a lot of content that needs to be updated regularly, or is the app mostly static?
        ::: tip Static vs. dynamic content
        For mostly static content, it's often enough to upload a new build from time to time. For dynamic content, you might need a CMS (content management system) or a database connection.
        :::
    - Which devices and browsers are the target audience using?
        ::: tip Browser compatibility and testing
        While Needle Engine works on all modern devices and browsers, it's always a good idea to test your app on the devices and browsers your target audience is using to make sure everything works as expected. For example, when creating an AR app for phones, you'll want to test across Android and iOS devices.
        :::

2. **Set up the project, a test deployment, and client deployment.**   
   It's often a good idea to test the deployment process early on, to make sure you understand how it works and what the requirements are. For example, when you've decided on using FTP, then you could set up a test folder on your own web server and test the deployment process there. Once changes are approved by the client, you can then deploy to the client's server. 

3. **Start creating!**  
   With requirements and deployment in place, go ahead and start making your project! You'll usually iterate locally, then deploy to your test server for approval, and then to the client's server.  

## Wordpress

1. Decide on the method you want to use to embed your Needle Engine project. You can either use the "Embedding a Needle project into an existing website" method, or the "Embedding a Needle project as iframe" method.

2. Upload the content of the `dist/` folder from your Web Project to your web hoster. Usually, the Wordpress uploads directory is located at `wp-content/uploads/`. 

    ::: tip Wordpress Backups
    You can decide if your new project should be at `wp-content/uploads/my-project/`, or at a different location like `my-projects/my-project`. This affects if and how your project will be contained in Wordpress backups.
    :::

3. In the page you want to add Needle Engine to, add a `HTML` block and paste the code snippet as outlined above – either as script embed, or as iframe.

## Shopify

::: warning <b>Under construction.</b> Needs to be documented.
:::

## Wix

::: warning <b>Under construction.</b> Needs to be documented.
:::

## Webflow

::: warning <b>Under construction.</b> Needs to be documented.
:::

---
title: Everywhere Actions
---

## What are Everywhere Actions?

Needle's Everywhere Actions are a set of carefully chosen components that allow you to create interactive experiences in Unity without writing a single line of code.  
They are designed to serve as building blocks for experiences across the web, mobile and XR, **including Augmented Reality on iOS**.  

From low-level triggers and actions, higher-level complex interactive behaviours can be built.  

### Supported Platforms
- Desktop
- Mobile (Android / iOS)
- VR Glasses
- AR Devices
- iOS AR – QuickLook (yes, really!)

## How do I use Everywhere Actions?

For iOS support add the `USDZExporter` component to your scene. It is good practice to add it to the same object as the `WebXR` component (but not mandatory)

To add an action to any object in your scene  
select it and then click `Add Component > Needle > Everywhere Actions > [Action]`.

![](/imgs/everywhere-actions-component-menu.gif)

## List of Everywhere Actions

| Action | Description | Example Use Cases |
| --- | --- | --- |
| Play Animation on Click | Plays a selected animation state from an Animator. After playing, it can optionally transition to another animation. | Product presentations, interactive tutorials, character movement |
| Change Material on Click | Switch out one material for others. All objects with that material will be switched together. | Product configurators, characters |
| Look At | Make an object look at the camera. | UI elements, sprites, info graphics, billboard effects, clickable hotspots |
| Play Audio on Click | Plays a selected audio clip. | Sound effects, Narration, Museum exhibits |
| Hide on Start | Hides an object at scene start for later reveal. |
| Set Active on Click | Show or hide objects. |  |
| Change Transform on Click | Move, rotate or scale an object. Allows for absolute or relative movement. | Characters, products, UI animation (use animation for more complex movements) |
| Audio Source | Plays audio on start and keeps looping. Spatial or non-spatial | Background music, ambient sounds |
| WebXR Image Tracking | Tracks an image target and shows or hides objects. | AR experiences, product presentations |

## Samples

### Musical Instrument

Demonstrates spatial audio, animation, and interactions.

<sample src="https://engine.needle.tools/samples-uploads/musical-instrument" />

### Simple Character Controllers

Demonstrates combining animations, look at, and movement.  

<sample src="https://engine.needle.tools/samples-uploads/usdz-characters" />

### Image Tracking

Demonstrates how to attach 3D content onto a custom image marker. Start the scene below in AR and point your phone's camera at the image marker on a screen, or print it out.     

<img src="https://engine.needle.tools/samples-uploads/image-tracking/assets/needle-marker.png" alt="Image Marker" width=300 />    

<a href="https://engine.needle.tools/samples-uploads/image-tracking/assets/needle-marker.png" target="_blank">Download Sample Image Marker</a>  

**On Android:** please turn on "WebXR Incubations" in the Chrome Flags. You can find those by pasting [chrome://flags/#webxr-incubations](chrome://flags/#webxr-incubations) into the Chrome browser address bar of your Android phone.  

<sample src="https://engine.needle.tools/samples-uploads/image-tracking" />

### Interactive Building Blocks Overview

<sample src="https://engine.needle.tools/samples-uploads/usdz-interactivity" />

## Create your own Everywhere Actions

Creating new Everywhere Actions involves writing code for your action in TypeScript, which will be used in the browser and for WebXR, and using our TriggerBuilder and ActionBuilder API to create a matching setup for Augmented Reality on iOS via QuickLook. When creating custom actions, keep in mind that QuickLook has a limited set of features available. You can still use any code you want for the browser and WebXR, but the behaviour for QuickLook may need to be an approximation built from the available triggers and actions. 

:::tip
Often constructing specific behaviours requires thinking outside the box and creatively applying the available low-level actions. An example would be a "Tap to Place" action – there is no raycasting or hit testing available in QuickLook, but you could cover the expected placement area with a number of invisible objects and use a "Tap" trigger to move the object to be placed to the position of the tapped invisible object.  
:::

Triggers and Actions for QuickLook are based on [Apple's Preliminary Interactive USD Schemas](https://developer.apple.com/documentation/arkit/usdz_schemas_for_ar/actions_and_triggers)

### Code Example

Here's the implementation for `HideOnStart` as an example for how to create an Everywhere Action with implementations for both the browser and QuickLook:  
@[code ts twoslash](@code/component-everywhere-action-hideonstart.ts)

::: tip
Often, getting the right behaviour will involve composing _higher-level actions_ from the available _lower-level actions_. For example, our "Change Material on Click" action is composed of a number of `fadeActions` and internally duplicates objects with different sets of materials each. By carefully constructing these actions, complex behaviours can be achieved.  
:::

### Low level methods for building your own actions

| Triggers | |
| --- | --- |
| `TriggerBuilder.sceneStartTrigger` | |
| `TriggerBuilder.tapTrigger` | |

| Actions | |
| --- | --- |
| `ActionBuilder.fadeAction` | |
| `ActionBuilder.startAnimationAction` | |
| `ActionBuilder.waitAction` | |
| `ActionBuilder.lookAtCameraAction` | |
| `ActionBuilder.emphasize` | |
| `ActionBuilder.transformAction` | |
| `ActionBuilder.playAudioAction` | |

|  Group Actions | |
| --- | --- |
| `ActionBuilder.sequence` | |
| `ActionBuilder.parallel` | |
| `GroupAction.addAction` | |
| `GroupAction.makeParallel` | |
| `GroupAction.makeSequence` | |
| `GroupAction.makeLooping` | |
| `GroupAction.makeRepeat` | |

To see the implementation of our built-in Everywhere Actions, please take look at `src/engine-components/export/usdz/extensions/behavior/BehaviourComponents.ts`.


## Further reading 

The following pages provide more examples and samples that you can test and explore right now:

- Visit our [AR Showcase Website](https://engine.needle.tools/projects/ar-showcase/) that has many interactive AR examples with a focus on iOS AR & Quicklook
- [Needle Engine Everywhere Action Samples](https://engine.needle.tools/samples/?overlay=samples&tag=everywhere+actions)

# Example Projects ✨

Explore some real world applications, websites and demos made with Needle Engine.

<p align="left">
<a href="./getting-started/">Get started now</a> • <a href="./vision">Learn more about our vision</a> • <a href="https://fwd.needle.tools/needle-engine/docs/features-overview">Features Overview</a> • <a href="https://docs.needle.tools/samples?utm_source=needle_docs">Samples to download</a>
</p>


## Needle Website  
<a href="https://needle.tools?utm_source=needle_docs" target="_blank">Visit Website</a> — by Needle   

<video-embed src="https://user-images.githubusercontent.com/5083203/186126996-27b45c5f-f3b9-40f7-b8c7-6ecba1d25a6e.mp4"/>


## Castle Builder
[Play Now](https://castle.needle.tools) — by Needle   

<video-embed src="https://user-images.githubusercontent.com/5083203/186145731-705cfec2-1779-4a0b-97d9-95f3edaaf2d0.mp4"/>




## Bike Configurator
[Bike Configurator](https://bike.needle.tools) — by Needle  

<video-embed src="https://user-images.githubusercontent.com/5083203/186146814-52fb05c7-a073-4efa-a226-47a9c1835413.mp4"/>


## Sandbox Template
[Sandbox Template](https://fwd.needle.tools/needle-engine/glitch-starter) — by Needle   

<video-embed src="https://user-images.githubusercontent.com/5083203/186149117-ca7cf22f-dc7d-4c74-86d4-d78fe53a208c.mp4"/>


## Songs of Cultures  
[Songs of Cultures](https://fwd.needle.tools/needle-engine/projects/songs-of-cultures) — by A.MUSE  

<video-embed src="https://user-images.githubusercontent.com/5083203/186147814-159a33f9-f1a6-47d4-804f-5f8f5a63125d.mp4"/>

## Pokémon Card
[Pokémon Card](https://fwd.needle.tools/needle-engine/projects/pokemon-card) — Scene from Alex Ameye • [Original Blog Post by Alex](https://alexanderameye.github.io/notes/holographic-card-shader/)  

<video-embed src="https://user-images.githubusercontent.com/5083203/186149736-49a697b3-4282-4b71-ab13-a6b176955c13.mp4"/>


## Encryption in Space  
[Encryption in Space](https://fwd.needle.tools/needle-engine/projects/encryption) — by Katja Rempel & Nick Jwu  

<video-embed src="https://user-images.githubusercontent.com/5083203/186151157-0c0a7d05-ad42-44be-b553-8d4cd48cbb81.mp4"/>

## Physics Playground
[Physics Playground](https://bruno-simon-20k-needle.glitch.me/) — Scene from Bruno Simon  

<video-embed src="https://user-images.githubusercontent.com/5083203/186149536-987ee796-3fe0-42bc-bd80-4c25aaf174aa.mp4"/>

---
title: Exporting Assets to glTF
---



# Exporting Assets, Animations, Prefabs, Materials, Lightmaps...
Add an ``ExportInfo`` component to your Unity scene to generate a new web project from a template, link to an existing web project that you want to export to, set up dependencies to other libraries and packages and to deploy your project. 

By default, your scene is exported on save. This setting can be changed by disabling ``Auto Export`` in the ``ExportInfo`` component.  

## 📦 Exporting glTF files 
To export meshes, materials, animations, textures (...) create a new GameObject in your hierarchy and add a ``GltfObject`` component to it. This is the root of a new glTF file. It will be exported whenever you make a change to the scene and save.  

Only scripts and data on and inside those root objects is exported. Scripts and data outside of them are not exported. 


Add a cube as a child of your root object and save your scene. Note that the output ``assets/`` folder (see [project structure](#vite-project-structure)) now contains a new ``.glb`` file with the same name as your root GameObject.  

You can enable the ``Smart Export`` setting (via `Edit/Project Settings/Needle` ) to only export when a change in this object's hierarchy is detected. 

:::details How to prevent specific objects from being exported
Objects with the `EditorOnly` tag will be ignored on export including their child hierarchy.   
Be aware that this is preferred over disabling objects as disabled will still get exported in case they're turned on later.
:::

### Lazy loading and multiple levels / scenes

If you want to split up your application into multiple levels or scenes then you can simply use the `SceneSwitcher` component. You can then structure your application into multiple scenes or prefabs and add them to the SceneSwitcher array to be loaded and unloaded at runtime. This is a great way to avoid having to load all your content upfront and to keep loading times small (for example it is what we did on [needle.tools](https://needle.tools?utm_source=needle_docs&utm_content=export_scenes) by separating each section of your website into its own scene and only loading them when necessary)

### Recommended Complexity per glTF

- Max. 50 MB export size uncompressed (usually ends up ~10-20 MB compressed)  
- Max. 500k vertices (less if you target mobile VR as well)  
- Max. 4x 2k lightmaps  

You can split up scenes and prefabs into multiple glTF files, and then load those on demand (only when needed). This keeps loading performance fast and file size small. See the [AssetReference section in the Scripting docs](scripting.md#assetreference-and-addressables).

 The scene complexity here is recommended to ensure good performance across a range of web-capable devices and bandwidths. There's no technical limitation to this beyond the capabilities of your device.

### Prefabs
Prefabs can be exported as invidual glTF files and instantiated at runtime. To export a prefab as glTF just reference a prefab asset (from the project browser and not in the scene) [from one of your scripts](https://fwd.needle.tools/needle-engine/docs/addressables).  

Exporting Prefabs works with nesting too: a component in a Prefab can reference another Prefab which will then also be exported.  
This mechanism allows for composing scenes to be as lightweight as possible and loading the most important content first and defer loading of additional content.  

### Scene Assets
Similar to Prefab assets, you can reference other Scene assets.  
To get started, create a component in Unity with a ``UnityEditor.SceneAsset`` field and add it to one of your GameObjects inside a GltfObject. The referenced scene will now be exported as a separate glTF file and can be loaded/deserialized as a ``AssetReference`` from TypeScript.  

You can keep working inside a referenced scene and still update your main exporter scene/website. On scene save or play mode change we will detect if the current scene is being used by your currently running server and then trigger a re-export for only that glb. (This check is done by name - if a glb inside your ``<web_project>/assets/`` folder exists, it is exported again and the main scene reloads it.)

As an example on [our website](https://needle.tools?utm_source=needle_docs&utm_content=export_sceneassets) each section is setup as a separate scene and on export packed into multiple glb files that we load on demand:

![2022-08-22-172605_Needle_Website_-_Website_-_Windows,_Mac,_Linux_-_U](https://user-images.githubusercontent.com/5083203/185958983-71913c97-5eec-4cfd-99f5-76798582373e.png)

#### Loading a Prefab or Scene from a custom script
If you want to reference and load a prefab from one of your scripts you can declare a `AssetReference` type.  
Here is a minimal example: 

@[code ts twoslash](@code/component-prefab.ts)

## 🏇 Exporting Animations 
Needle Engine supports a considerable and powerful subset of Unity's animation features:

- **Timeline** incl. activation tracks, animation tracks, track offsets
- **Animator** incl. top-level state transitions
  - Blend trees are currently not supported.
  - Sub state machines are currently not supported.
- **AnimationClips** incl. Loop modes
- **Procedural Animations** can be created via scripting

Needle Engine is one of the first to support the new [glTF extension KHR_ANIMATION_POINTER](https://github.com/ux3d/glTF/tree/extensions/KHR_animation_pointer/extensions/2.0/Khronos/KHR_animation_pointer).  
This means that almost all properties, including script variables, are animatable.  

One current limitation is that materials won't be duplicated on export — if you want to animate the same material with different colors, for example, you currently need to split the material in two. 

## 🌍 Exporting the Skybox 
The Unity skybox and custom reflection (if any) are baked into a texture on export and automatically exported inside the ``NEEDLE_lightmaps`` extension.  

To change the skybox resolution you can add a ``SkyboxExportSettings`` component to your scene.  

![image](https://user-images.githubusercontent.com/5083203/196030839-170a9496-9ed9-4ebc-bc1d-2df6c746f8c8.png)


If you don't want to skybox to be exported at all in a glb file you can untick the ``Embed Skybox`` option on your ``GltfObject`` component  

![image](https://user-images.githubusercontent.com/5083203/196030825-8a05037f-5acc-4795-9128-2bdacedd0d49.png)


## ✨ Exporting Materials

### Physically Based Materials (PBR)
By default, materials are converted into glTF materials on export. glTF supports a physically based material model and has a number of extensions that help to represent complex materials.  

For full control over what gets exported, it's highly recommended to use the glTF materials provided by UnityGltf:
- PBRGraph
- UnlitGraph

::: tip When in doubt, use the PBRGraph shader
The PBRGraph material has a lot of features, way more than Standard or URP/Lit. These include advanced features like refraction, iridescence, sheen, and more. Additionally, materials using PBRGraph and UnlitGraph are exported as-is, with no conversion necessary. 
:::

Materials that can be converted out-of-the-box:
- BiRP/Standard
- BiRP/Autodesk Interactive
- BiRP/Unlit
- URP/Lit
- URP/Unlit

Other materials are converted using a propery name heuristic. That means that depending on what property names your materials and shaders use, you might want to either refactor your custom shader's properties to use the property names of either URP/Lit or PBRGraph, or export the material as [Custom Shader](#custom-shaders).

### Custom Shaders
To export custom unlit shaders (for example made with ShaderGraph), add an ``ExportShader`` Asset Label to the shader you want to export. Asset Labels can be seen at the bottom of the Inspector window.

![2022-08-22-172029_Needle_Website_-_CustomShaders_-_Windows,_Mac,_Lin](https://user-images.githubusercontent.com/5083203/185957781-9fae18c5-09ff-490f-8958-57e138aa0003.png)

#### Limitations
- We currently only support custom **Unlit** shaders — Lit shader conversion is not officially supported.
- Custom Lit Shaders are currently experimental. Not all rendering modes are supported. 
- Shadow receiving on custom shaders is not supported
- Skinned meshes with custom shaders are not supported
- As there's multiple coordinate system changes when going from Unity to three.js and glTF, there might be some changes necessary to get advanced effects to work. We try to convert data on export but may not catch all cases where conversions are necessary.  
  - UV coordinates in Unity start at the bottom left; in glTF they start at the top left.
  - X axis values are flipped in glTF compared to Unity. This is a variant of a left-handed to right-handed coordinate system change. Data used in shaders may need to be flipped on X to display correctly.  

::: note Not part of the glTF specification
Note that **Custom Shaders** aren't officially part of the glTF specification. Our implementation of custom shaders uses an extension called KHR_techniques_webgl, that stores the WebGL shader code directly in the glTF file. The resulting assets will work in viewers based on Needle Engine, but may not display correctly in other viewers.
:::

## 💡 Exporting Lightmaps 
![2022-08-22-171650_Needle_-_Google_Chrome](https://user-images.githubusercontent.com/5083203/185957005-d04c9530-07eb-40f5-b305-9822d13b79ab.png)


To export lightmaps simply [generate lightmaps](https://docs.unity3d.com/Manual/Lightmapping.html) in Unity. Lightmaps will be automatically exported.

When working on multiple scenes, disable "Auto Generate" and bake lightmaps explicitly. Otherwise, Unity will discard temporary lightmaps on scene change.

### Recommended Lightmap Settings
- Lightmap Encoding: Normal Quality (adjust in Project Settings > Player)
- Progressive GPU (faster and usually accurate enough for small scenes)
- Non-Directional Lightmaps
- Max Lightmap Size 2k (you can go higher, but expect large files)
- Max 4x 2k lightmaps per scene (you can go higher, but expect large files)
- Compress Lightmaps OFF (increases quality; otherwise will be compressed again at export time)

![2022-08-22-171356_Needle_Website_-_Lightmaps_-_Windows,_Mac,_Linux_-](https://user-images.githubusercontent.com/5083203/185956392-f4031f45-ad13-4e6d-a14c-c8ec5c1fcfd4.png)

### Mixing Baked and Non-Baked Objects

There's no 100% mapping between how Unity handles lights and environment and how three.js handle that. For example, Unity has entirely separate code paths for lightmapped and non-lightmapped objects (lightmapped objects don't receive ambient light since that is already baked into their maps), and three.js doesn't distinguish in that way.  

This means that to get best results, we currently recommend specific settings if you're mixing baked and non-baked objects in a scene:  
```
Environment Lighting: Skybox
Ambient Intensity: 1
Ambient Color: black
```

**2021.3+**  
![20220826-175324-SqBL-Unity_pMXa-needle](https://user-images.githubusercontent.com/2693840/186947184-2446672f-420c-47e8-8f7d-970a7d52bf35.png)

**2020.3+**  
![20220826-175514-tnGc-Unity_mycs-needle](https://user-images.githubusercontent.com/2693840/186947203-2d7d96c3-f566-44b4-889c-4103fac505d4.png)

If you have no baked objects in your scene, then the following settings should also yield correct results:  
```
Environment Lighting: Color
Ambient Color: any
```

---
title: Questions and Answers (FAQ) 💡
---


## How can I activate my Needle Engine License?

### Activating the license in Unity

#### Needle Engine 4.x

Go to Project Settings/Needle and click on the login button. Follow the steps and log in to your Needle account.   
After that you'll see your account information in the Unity project settings window. Select the licensed team from the dropdown.   

#### Needle Engine 3.x

Open `Edit/Project Settings/Needle` to get the Needle Engine plugin settings. At the top of the window you'll find fields for entering your license information.
- `Email` - Enter the email you purchased the license with
- `Invoice ID` - Enter one of the invoice ids that you received by email

Note: You might need to restart the local webserver to apply the license.

![unity license window](/imgs/unity-needle-engine-license.jpg)

### Activating the license in Blender
Open `Addon Preferences/Needle Engine` to get to the Needle Engine addon settings 
- `Email` - Enter the email you purchased the license with
- `Invoice ID` - Enter one of the invoice ids that you received by email

Note: You might need to restart the local webserver to apply the license.



## My local website shows an SSL error e.g. 'Your connection is not private'

You might see a warning in your browser about SSL Security depending on your local configuration.  

This is because while the connection is encrypted, by default there's no SSL certificate that the browser can validate.
If that happens: click `Advanced` and `Proceed to Site`. In Safari, you might need to refresh the page afterwards, because it does not automatically proceed. Now you should see your scene in the browser!  

The dialogue should only show up once for the same local server

::: tip
Connections are secured, because we're enforcing HTTPS to make sure that WebXR and other modern web APIs work out-of-the-box. Some browsers will still complain that the SSL connection (between your local development server and the local website) can't be automatically trusted, and that you need to manually verify you trust that page. Automatic Page Reload and Websocket connections may also be affected depending on the browser and system settings.  

See [the Testing docs](./testing.md) for information on how to set up a self-signed certificate for a smoother development experience.
:::

![SLL warning on chrome](/videos/ssl-warning.gif)
  
  

## My local website stays black

If that happens there's usually an exception either in engine code or your code. Open the dev tools (<kbd>Ctrl + Shift + I</kbd> or <kbd>F12</kbd> in Chrome) and check the Console for errors.  
In some cases, especially when you just updated the Needle Engine package version, this can be fixed by stopping and restarting the local dev server.  
For that, click on the running progress bar in the bottom right corner of the Editor, and click the little <kbd>X</kbd> to cancel the running task. Then, simply press Play again.  


## My objects are white after export
This usually happens when you're using custom shaders or materials and their properties don't cleanly translate to known property names for glTF export.  
You can either make sure you're using glTF-compatible materials and shaders, or mark shaders as "custom" to export them directly.  
- Read more about recommended glTF workflows: <link>
- Read more about custom shaders: <link>


## Uncaught ReferenceError: NEEDLE_ENGINE_META is not defined / NEEDLE_USE_RAPIER is not defined

If you are using vite or next.js make sure to add the Needle Engine plugins to your config.
Example for vite:
```js
const { needlePlugins } = await import('@needle-tools/engine/plugins/vite/index.js');
plugins: [needlePlugins(command, needleConfig)]
```
Example for next.js
```js
const { needleNext } = await import("@needle-tools/engine/plugins/next/index.js");
return needleNext({}, { modules: { webpack } });
```
You can also just declare the missing variables in e.g. your root `index.html` in a script tag like so:
```html
<script>
  var NEEDLE_ENGINE_META = {}
  var NEEDLE_USE_RAPIER = true;
</script>
```

## THREE.EXRLoader: provided file doesnt appear to be in OpenEXR format

Please make sure that sure that you have set Lightmap Encoding to **Normal Quality**.   
Go to *Edit/Project Settings/Player* for changing the setting.  

![](/faq/lightmap_encoding.jpg)  

## My website becomes too large / is loading slow (too many MB)
  
This can have many reasons, but a few common ones are:
- too many textures or textures are too large
- meshes have too many vertices
- meshes have vertex attributes you don't actually need (e.g. have normals and tangents but you're not using them)
- objects are disabled and not ignored – disabled objects get exported as well in case you want to turn them on at runtime! Set their Tag to `EditorOnly` to completely ignore them for export.
- you have multiple ``GltfObject`` components in your scene and they all have ``EmbedSkybox`` enabled (you need to have the skybox only once per scene you export)
  
If loading time itself is an issue you can **try to split up your content into multiple glb files** and load them on-demand (this is what we do on our website). For it to work you can put your content into Prefabs or Scenes and reference them from any of your scripts. Please have a look at [Scripting/Addressables in the documentation](./scripting.md#assetreference-and-addressables).

## My UI is not rendering Text

- For Unity: Make sure that you use the `UI/Legacy/Text` component and **not** the `TextMeshPro - Text` component

## My scripts don't work after export
  
- Your existing C# code will *not* export as-is, you have to write matching typescript / javascript for it.  
- Needle uses typescript / javascript for components and generates C# stubs for them.  
- Components that already have matching JS will show that in the Inspector.  

## My lightmaps look different / too bright

Ensure you're following [best practices for lightmaps](https://docs.needle.tools/lightmaps?utm_source=needle_docs) and read about [mixing baked and non-baked objects](https://github.com/needle-tools/needle-engine-support/blob/main/documentation/export.md#mixing-baked-and-non-baked-objects)

## My scene is too bright / lighting looks different than in Unity
Make sure that your lights are set to "Baked" or "Realtime". "Mixed" is currently not supported.  
  
- Lights set to mixed (with lightmapping) do affect objects twice in three.js, since there is currently no way to exclude lightmapped objects from lighting
- The ``Intensity Multiplier`` factor for Skybox in ``Lighting/Environment`` is currently not supported and has no effect in Needle Engine  
  ![image](https://user-images.githubusercontent.com/5083203/185429006-2a5cd6a1-8ea2-4a8e-87f8-33e3afd080ec.png)
- Light shadow intensity can currently not be changed due to a three.js limitation.
  
Also see the docs on [mixing baked and non-baked objects](https://github.com/needle-tools/needle-engine-support/blob/main/documentation/export.md#mixing-baked-and-non-baked-objects).


## My skybox resolution is low? How to change my skybox resolution

  - **If you use a custom cubemap**: You can override the texture import settings of the skybox texture (assigned to your cubemap)
  
    ![image](https://user-images.githubusercontent.com/5083203/188179104-1e078cda-3397-4ebe-aaf9-7faa23ee4904.png)   

 
  - **If you use the default skybox**: Add a ``SkyboxExportSettings`` component anywhere in your scene to override the default resolution   
  
    ![image](https://user-images.githubusercontent.com/5083203/188171443-578380ab-2036-4d70-a8a7-f8cd9da9f603.png)



## My Shadows are not visible or cut off 
  
Please the following points:   
  
  - Your light has shadows enabled (either Soft Shadow or Hard Shadow)
  - Your objects are set to "Cast Shadows: On" (see MeshRenderer component)
  - For directional lights the position of the light is currently important since the shadow camera will be placed where the light is located in the scene.



## My colors look wrong
  
Ensure your project is set to Linear colorspace.

![image](https://user-images.githubusercontent.com/5083203/191774978-66e9feb1-0551-4549-85d3-3e5b8021f162.png)



## I'm using networking and Glitch and it doesn't work if more than 30 people visit the Glitch page at the same time
  
- Deploying on Glitch is a fast way to prototype and might even work for some small productions. The little server there doesn't have the power and bandwidth to host many people in a persistent session.  
- We're working on other networking ideas, but in the meantime you can host the website somewhere else (with node.js support) or simply remix it to distribute load among multiple servers. You can also host the [networking backend package](https://www.npmjs.com/package/@needle-tools/needle-tiny-networking-ws) itself somewhere else where it can scale e.g. Google Cloud.



## My website doesn't have AR/VR buttons
  
- Make sure to add the `WebXR` component somewhere inside your root `GltfObject`.
- Optionally add a `AR Session Root` component on your root `GltfObject` or within the child hierarchy to specify placement, scale and orientation for WebXR.
- Optionally add a `XR Rig` component to control where users start in VR


## I created a new script in a sub-scene but it does not work
  When creating new scripts in npmdefs in sub-scenes (that is a scene that is exported as a reference from a script in your root export scene) you currently have to re-export the root scene again. This is because the code-gen that is responsible for registering new scripts currently only runs for scenes with a ``ExportInfo`` component. This will be fixed in the future.


## My local server does not start / I do not see a website
  
The most likely reason is an incorrect installation. 
Check the console and the `ExportInfo` component for errors or warnings.  

If these warnings/errors didn't help, try the following steps in order. Give them some time to complete. Stop once your problem has been resolved. Check the console for warnings and errors.  
  
- Make sure you follow the [Prerequisites](./getting-started/#prerequisites).
- Install your project by selecting your `ExportInfo` component and clicking `Install` 
- Run a clean installation by selecting your `ExportInfo` component, holding Alt and clicking `Clean Install`
- Try opening your web project directory in a command line tool and follow these steps:
  - run ``npm install`` and then ``npm run dev-host``
  - Make sure both the local runtime package (``node_modules/@needle-tools/engine``) as well as three.js (``node_modules/three``) did install. 
  - You may run ``npm install`` in both of these directories as well.


## Does C# component generation work with javascript only too?
  While generating C# components does technically run with vanilla javascript too we don't recommend it and fully support it since it is more guesswork or simply impossible for the generator to know which C# type to create for your javascript class. Below you find a minimal example on how to generate a Unity Component from javascript if you really want to tho.    
  
```js
import { Behaviour } from "@needle-tools/engine";

export class MyScript extends Behaviour
{
    //@type float
    myField = 5;
}
```


## I don't have any buttons like "Generate Project" in my components/inspector
  
Please check that you're not accidentally in the Inspector's `Debug` mode – switch back to `Normal`:  
![20220824-025011-S2GQ-Unity_lKlT-needle](https://user-images.githubusercontent.com/2693840/186291615-56e7ebdb-1221-4326-813d-f88526fa126c.png)


## Toktx can not be found / toktx is not installed

- Make sure to [download and install toktx](http://localhost:8080/docs/getting-started/.html#install-these-tools-for-production-builds)

- On Windows: Make sure you have added toktx to your system environment variables. You may need to restart your computer after adding it to refresh the environment variables. The default install location is ``C:\Program Files\KTX-Software\bin``   

![image](/imgs/ktx-env-variable.webp)


## Installing the web project takes forever / does never finish / EONET: no such file or directory
- **Make sure to not create a project on a drive formatted as exFAT** because exFAT does not support symlinks, which is required for Needle Engine for Unity prior to version 3.x.    
  You can check the formatting of your drives using the following steps:
  1. Open "System Information" (either windows key and type that or enter "msinfo32" in cmd)
  2. Select Components > Storage > Drives
  3. Select all (Ctrl + A) on the right side of the screen and copy that (<kbd>Ctrl + C</kbd>) and paste here (<kbd>Ctrl + V</kbd>)

## NPM install fails and there are errors about hard drive / IO
Make sure your project is on a disk that is known to work with node.js. Main reason for failures is that the disk doesn't support symlinks (symbolic links / softlinks), which is a requirement for proper functioning of node.js.  
<kbd>NTFS</kbd> formatting should always work. Known problematic file system formattings are <kbd>exFAT</kbd> and <kbd>FAT32</kbd>. 

To check the format of your drives, you can:
1. Open "System Information" (either <kbd>Windows key</kbd> and type "System Information" or enter `msinfo32` in cmd <kbd>Windows + R</kbd>)
2. Select "Components > Storage > Drives"
3. There, you can see all drives and their formatting listed. Put your projects on a drive that is NTFS formatted.


## I'm getting errors with "Unexpected token `@`. Expected identifier, string literal, numeric literal or ..."

Needle Engine uses typescript decorators for serialization.   
To fix this error make sure to enable `experimentalDecorators` in your tsconfig.json 

## I'm getting an error 'failed to load config ... vite.config.js' when running npm commands on Mac OS

You're likely using an x86_64 version of Unity on an (ARM) Apple Silicon processor. Unity 2020.3 is only available for x86_64, later versions also have Apple Silicon versions.  
Our Unity integration calling npm will thus do so from an x86_64 process, resulting in the x86_64 version of node and vite/esbuild being used. When you afterwards try to run npm commands in the same project from an Apple Silicon app (e.g. VS Code), npm will complain about mismatching architectures with a long error message.  

To fix this, use an Apple Silicon version of Unity (2021.1 or later).  

You can also temporarily fix it on 2020.3 by deleting the `node_modules` folder and running `npm install` again from VS Code. You'll have to delete `node_modules` again when you switch back to Unity.

## Circular reference error

This can happen when you have e.g. a `SceneSwitcher` (or any other component that loads a scene or asset) and the referenced Asset in Unity contains a `GltfObject` that has the same name as your original scene with the `SceneSwitcher`. You can double check this in Unity if you get an error that says something like:

```
Failed to export ↑ YourSceneName.glb
you seem to have objects with the same name referencing each other.
```

To fix this you can:
- Remove the `GltfObject` in the referenced Prefab or Scene
- Rename the GameObject with the component that loads the referenced scenes

If this doesn't fix the problem please ask [in our forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content).

## My scene is not loading and the console contains a warning with 'circular references' or 'failed to update active state'
Please see the [circular reference error](#circular-reference-error) section.

## Does my machine support WebGL 2?

Use a detector [like this one](https://get.webgl.org/webgl2/) to determine if your device supports WebGL 2, it also hints at what could be the cause of your problem, but generally make sure you have updated your browser and drivers. WebGL 1 is not supported.

#### Known devices to cause issues:
- Lenovo Thinkpad - T495  

## I want to use Needle AI with my local AI model

If you want (or have to) run your AI locally you can use the Needle llms.txt files as context for your local AI (e.g. Ollama):

- [llms.txt](https://cloud.needle.tools/llms.txt)
- [llms-full.txt](https://cloud.needle.tools/llms-full.txt)


## Still have questions? 
[Ask in our forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content) 

<a href="https://discord.needle.tools" target="_blank"><img height=20 src="https://img.shields.io/discord/717429793926283276?color=5562ea&label=Discord" /></a>

# Feature Overview

Needle Engine is a fully fledged 3D engine that runs in the browser. It comes with all the features you'd expect from a modern 3D engine, and more.  If you haven't yet, take a look at our [Homepage](https://needle.tools) and our [Samples and Showcase](https://engine.needle.tools/samples).

[[toc]]

## Shaders and Materials

Both PBR Materials and Custom shaders created with Shader Graph or other systems can be exported. 

<img src="https://user-images.githubusercontent.com/5083203/186012027-9bbe3944-fa56-41fa-bfbb-c989fa87aebb.png" />

Use the node based [ShaderGraph](https://unity.com/features/shader-graph) to create shaders for the web. ShaderGraph makes it easy for artists to keep creating without having to worry about syntax.

Read more about [PBR Materials](./export.md#physically-based-materials-pbr) • [Custom Shaders](./export.md#custom-shaders)

## Crossplatform: VR, AR, Mobile, Desktop  
Needle Engine runs everywhere web technology does: run the same application on desktop, mobile, AR or VR. We build Needle Engine [with XR in mind](./xr.md) and consider this as and integral part of responsive webdesign!

Use [Everywhere Actions](./everywhere-actions.md) for **Interactive AR on both Android and iOS**.


## Lightmaps

![lightmaps](https://user-images.githubusercontent.com/5083203/186163693-093c7ae2-96eb-4d75-b98f-bf19f78032ff.gif)

Lightmaps can be baked in Unity or Blender to easily add beautiful static light to your 3d content. Lightbaking for the web was never as easy. Just mark objects that you want to lightmap as static in Unity, add one or many lights to your scene (or use emissive materials) and click bake. Needle Engine will export your lightmaps per scene and automatically load and display them just as you see it in the Editor! 

> **Note**: There is no technical limitation on which lightmapper to use, as long as they end up in Unity's lightmapping data structures. Third party lightmappers such as [Bakery](https://assetstore.unity.com/packages/tools/level-design/bakery-gpu-lightmapper-122218) thus are also supported. 

- Read more about [Exporting Lightmaps](https://fwd.needle.tools/needle-engine/docs/lightmaps)

## Multiplayer and Networking
Networking is built into the core runtime. Needle Engine deployments to Glitch come with a tiny server that allows you to deploy a multiplayer 3D environment in seconds. The built-in networked components make it easy to get started, and you can create your own synchronized components. Synchronizing variables and state is super easy!  

- Read more about [Networking](https://fwd.needle.tools/needle-engine/docs/networking) • [Scripting](https://fwd.needle.tools/needle-engine/docs/scripting)

## Animations and Sequencing
Needle Engine brings powerful animations, state control and sequencing to the web — from just playing a single animation to orchestrating and blending complex animations and character controllers. The Exporter can translate Unity components like Animator and Timeline into a web-ready format.   
We even added this functionality to our Blender addon so you can craft compatible animation state machines and export nla tracks as timelines to the web from within Blender too.  

- Read more about [Animation Components](./component-reference.md#animation)

### Animator

<img src="https://user-images.githubusercontent.com/5083203/186011302-176524b3-e8e5-4e6e-9b77-7faf3561bb15.png" />

The [Animator and AnimatorController](https://docs.unity3d.com/Manual/class-AnimatorController.html) components in Unity let you setup animations and define conditions for when and how to blend between them. We support exporting state machines, StateMachineBehaviours, transitions and layers. StateMachineBehaviours are also supported with ``OnStateEnter``, ``OnStateUpdate`` and ``OnStateExit`` events.  


> **Note**: Sub-states and Blend Trees are not supported.  


### Timeline

![2022-08-23-013517_Scene](https://user-images.githubusercontent.com/5083203/186037829-ee99340d-b19c-484d-b551-94797519c9d9.png)

We're also translating [Unity's Timeline](https://unity.com/features/timeline) setup and tracks into a web-ready format.  
Supported tracks include: AnimationTrack, AudioTrack, ActivationTrack, ControlTrack, SignalTrack.   

> **Note**: Sub-Timelines are currently not supported.  

> **Note**: It's possible to [export custom timeline tracks](https://github.com/needle-tools/needle-engine-modules/tree/main/package/TimelineHtml).  

- Read more about [Animation Components](./component-reference.md#animation)

## Physics
Use Rigidbodies, Mesh Colliders, Box Colliders and SphereColliders to add some juicy physics to your world.

- Read more about [Physics Components](./component-reference.md#physics)

<sample src="https://engine.needle.tools/samples-uploads/physics-animation/" />

## UI
Building UI using Unity's UI canvas system is in development. Features currently include exporting Text (including fonts), Images, Buttons.  

See the [ui component reference](component-reference.md#ui) for supported component.

<sample src="https://engine.needle.tools/samples-uploads/screenspace-ui" />

## Particles
Export of Unity ParticleSystem (Shuriken) is in development. Features currently include world/local space simulation, box and sphere emitter shapes, emission over time as well as burst emission, velocity- and color over time, emission by velocity, texturesheet animation, basic trails.   
See a [live sample](https://engine.needle.tools/samples/particles) of supported features below:

<sample src="https://engine.needle.tools/samples-uploads/particles/" />

## PostProcessing  

Builtin effects include Bloom, Screenspace Ambient Occlusion, Depth of Field, Color Correction. You can also create your own custom effects. See [the component reference](./component-reference.md#postprocessing) for a complete list.  

<sample src="https://engine.needle.tools/samples-uploads/postprocessing/" />

## Editor Integrations
Needle Engine comes with powerful integrations into the Unity Editor and Blender.   
It allows you to setup and export complex scenes in a visual way providing easy and flexible collaboration between artists and developers. 

## Scripting
Needle Engine uses as [component based workflow](scripting.md#component-architecture). Create custom scripts in typescript or javascript. Use our [modular npm-based package workflow](https://fwd.needle.tools/needle-engine/docs/npmdef) integrated into Unity. A [typescript to C# component compiler](https://fwd.needle.tools/needle-engine/docs/component-compiler) produces Unity components magically on the fly. 

- Read more about [Scripting Reference](scripting) • [Npm Definition Files](https://fwd.needle.tools/needle-engine/docs/npmdef)


## And there is more

- PostProcessing → Bloom, Screenspace Ambient Occlusion, Depth of Field, Color Correction...
- EditorSync → Live synchronize editing in Unity to the running three.js application for local development
- Interactive AR on iOS and Android → Use our [Everywhere Actions](./everywhere-actions.md) feature set or build your own

---
# Where to go next

See our [Getting Started Guide](getting-started/) to learn about how to download and set up Needle Engine.   
Learn about [our vision](vision) or dive deeper into some of the [technical background and glTF](technical-overview) powering it all.

---
title: Scripting basics For Unity Developers
editLinks: false
sidebar: false
---

# This page has been moved: [continue here](./getting-started/for-unity-developers)

Moved to [Getting Started](./getting-started/)

---
title: Frameworks, Bundlers, HTML
---

## Bundling and web frontends

Needle Engine is build as a web component.   
This means just install `@needle-tools/engine` in your project and include `<needle-engine src="path/to/your.glb">` anywhere in your web-project.  

- Install using npm:   
  `npm i @needle-tools/engine`

With our default Vite based project template Needle Engine gets bundled into a web app on deployment. This ensures smaller files, tree-shaking (similar to code stripping in Unity) and optimizes load times. Instead of downloading numerous small scripts and components, only one or a few are downloaded that contain the minimal code needed.  

Vite (our default bundler) has a good explanation why web apps should be bundled: [Why Bundle for Production](https://vitejs.dev/guide/why.html)

### Vite, Vue, React, Svelte, React Three Fiber...

Needle Engine is unoponiated about the choice of framework. Our default template uses the popular [vite](https://vitejs.dev) as bundler. From there, you can add vue, svelte, nuxt, react, react-three-fiber or other frameworks, and we have samples for a lot of them. You can also integrate other bundlers, or use none at all – just plain HTML and Javascript.  

Here's some example tech stacks that are possible and that we use Needle Engine with:  

- **Vite + HTML** — This is what our default template uses!
    
- **Vite + Vue** — This is what the [Needle Tools](https://needle.tools) website uses!. Find a sample to download [here](https://github.com/needle-tools/needle-engine-samples). 
- **Vite + Svelte** 
- **Vite + SvelteKit**
- **Vite + React** — There's an experimental template shipped with the Unity integration for this that you can pick when generating a project!
- **react-three-fiber** — There's an experimental template shipped with the Unity integration for this that you can pick when generating a project!
- **Vercel & Nextjs** — Find a [example nextjs project here](https://github.com/needle-engine/nextjs-sample)
- **CDN without any bundler** — Find a code example [here](./vanilla-js.md)

In short: we're currently providing a minimal vite template, but you can extend it or switch to other frameworks –  
Let us know what and how you build, and how we can improve the experience for your usecase or provide an example!

:::tip
Some frameworks require custom settings in `needle.config.json`. Learn more [here](./reference/needle-config-json.md). Typically, the `baseUrl` needs to be set. 
:::

:::details How do I create a custom project template in Unity?

You can create and share your own web project templates to use other bundlers, build systems, or none at all.  

**Create a new Template**  
1. Select `Create/Needle Engine/Project Template` to add a ProjectTemplate into the folder you want to use as a template 
2. Done! It's that simple.

The dependencies come from unity when there is a NpmDef in the project (so when your project uses local references).  
You could also publish your packages to npm and reference them via version number.  
:::

### Tree-shaking to reduce bundle size

Tree shaking refers to a common practice when it comes to bundling of web applications ([see MSDN docs](https://developer.mozilla.org/en-US/docs/Glossary/Tree_shaking)). It means that code paths and features that are not used in your code will be removed from the final bundled javascript file(s) to reduce filesize. See below about features that Needle Engine includes and remove them: 

:::details How to remove Rapier physics engine? (Reduce the overall bundle size removing ~2MB (~600KB when gzipping))

- **Option 1**: via needlePlugins config:  
Set `useRapier` to `false` in your vite.config: `needlePlugins(command, needleConfig, { useRapier: false }),`

- **Option 2**: via vite.define config:  
  Declare the `NEEDLE_USE_RAPIER` define with `false`
  ```js
  define: {
    NEEDLE_USE_RAPIER: false
  },
  ```

- **Option 3**: via .env  
  Create a `.env` file in your web project and add `VITE_NEEDLE_USE_RAPIER=false`

- **Option 4**: via Unity component   
  Add the `Needle Engine Modules` component to your scene and set `Physics Engine` to `None`  
  ![](/imgs/unity-needle-engine-modules-physics.jpg)

:::

## Creating a PWA

We support easily creating a Progressive Web App (PWA) directly from our vite template.  
PWAs are web applications that load like regular web pages or websites but can offer user functionality such as working offline, push notifications, and device hardware access traditionally available only to native mobile applications.   

By default, PWAs created with Needle have offline support, and can optionally refresh automatically when you publish a new version of your app. 

1) Install the [Vite PWA plugin](https://vite-pwa-org.netlify.app/) in your web project: `npm install vite-plugin-pwa --save-dev`
2) Modify `vite.config.js` as seen below. Make sure to pass the same `pwaOptions` object to both `needlePlugins` and `VitePWA`.

```js
import { VitePWA } from 'vite-plugin-pwa';

export default defineConfig(async ({ command }) => {

    // Create the pwaOptions object. 
    // You can edit or enter PWA settings here (e.g. change the PWA name or add icons).
    /** @type {import("vite-plugin-pwa").VitePWAOptions} */
    const pwaOptions = {};
  
    const { needlePlugins } = await import("@needle-tools/engine/plugins/vite/index.js");

    return {
        plugins: [
            // pass the pwaOptions object to the needlePlugins and the VitePWA function
            needlePlugins(command, needleConfig, { pwa: pwaOptions }),
            VitePWA(pwaOptions),
        ],
        // the rest of your vite config...
```

:::tip All assets are cached by default
Note that by default, all assets in your build folder are added the PWA precache – for large applications with many dynamic assets, this may not be what you want (imagine the YouTube PWA caching all videos once a user opens the app!). See [More PWA Options](#more-pwa-options) for how to customize this behavior.  
:::

### Testing PWAs

To test your PWA, deploy the page, for example using the `DeployToFTP` component.  
Then, open the deployed page in a browser and check if the PWA features work as expected:
- the app shows up as installable
- the app works offline
- the app is detected as offline-capable PWA by [pwabuilder.com](https://pwabuilder.com/)

PWAs use Service Workers to cache resources and provide offline support. Service Workers are somewhat harder to use during development, and typically are only enabled for builds (e.g. when you use a `DeployTo...` component). 

You can enable PWA support for development by adding the following to the options object in your `vite.config.js`. 

```js
const pwaOptions = {
  // Note: PWAs behave different in dev mode. 
  // Make sure to verify the behaviour in production builds!
  devOptions: {
    enabled: true,
  }
};
```

Please note that PWAs in development mode do not support offline usage – trying it may result in unexpected behavior.  

### Automatically update running apps

Websites typically show new or updated content on page refresh. 

In some situations, you may want the page to refresh and reload automatically when a new version has been published – 
such as in a museum, trade show, public display, or other long-running scenarios.  

To enable automatic updates, set the `updateInterval` property in the pwaOptions object to a duration (in milliseconds) in which the app should check for updates. If an update is detected, the page will reload automatically.

```js
const pwaOptions = {
  updateInterval: 15 * 60 * 1000, // 15 minutes, in milliseconds
};
```

:::tip Periodic Reloads and User Data
It's not recommended to use automatic reloads in applications where users are interacting with forms or other data that could be lost on a reload. For these applications, showing a reload prompt is recommended.  
See the [Vite PWA plugin documentation](https://vite-pwa-org.netlify.app/guide/prompt-for-update.html) for more information on how to implement a reload prompt instead of automatic reloading.   
:::

### More PWA options

Since Needle uses the [Vite PWA plugin](https://vite-pwa-org.netlify.app/) under the hood, you can use all options and hooks provided by that.  
For example, you can provide a partial manifest with a custom app title or theme color:

```js
const pwaOptions = {
  // manifest options provided here will override the defaults 
  manifest: {
    name: "My App",
    short_name: "My App",
    theme_color: "#B2D464",
  }
};
```

For complex requirements like partial caching, custom service workers or different update strategies, you can remove the `{ pwa: pwaOptions }` option from `needlePlugins` and add PWA functionality directly through the Vite PWA plugin.

## Accessing Needle Engine and Components from external javascript
    
Code that you expose can be accessed from JavaScript after bundling. This allows to build viewers and other applications where there's a split between data known at edit time and data only known at runtime (e.g. dynamically loaded files, user generated content).  
For accessing components from regular javascript outside of the engine please refer to the [interop with regular javascript section](./scripting.md#accessing-needle-engine-and-components-from-anywhere)


## Customizing how loading looks

See the *Loading Display* section in [needle engine component reference](./reference/needle-engine-attributes.md)

### Builtin styles

The needle-engine loading appearance can use a light or dark skin.  
To change the appearance use the `loading-style` attribute on the `<needle-engine>` web component.  
Options are `light` and `dark` (default):

``<needle-engine loading-style="light"></needle-engine>``

### Custom Loading Style — *PRO feature*  #

Please see the *Loading Display* section in [needle engine component reference](./reference/needle-engine-attributes.md)

![custom loading](/imgs/custom-loading-style.webp)

---
home: false
next: ./getting-started/index.md
editLink: false
lastUpdated: false
footer: "Copyright © 2025 Needle Tools GmbH"
---

<discountbanner fallback_image="/docs/imgs/banner.webp" /> 

<quoteslides>

<testimonial
  name="Chris Mahoney"
  role="Founder, Our Computer Company. Ex-Unity, Ex-Lamina1"
  src="https://x.com/mahoneymatic/status/1562981022932684800?t=qNqojoZkk2CZrJa7dGzqng&s=19"
  img="https://pbs.twimg.com/profile_images/1872967967404642304/kOGAWAzd_x96.jpg"
>
Unbelievable Unity editor integration by an order of magnitude,  
  and as straightforward as the docs claim. Wow.
</testimonial>

<testimonial 
  name="Kevin Curry" 
  role="Founder, Intangible.ai. Ex-Unity"
  src="https://x.com/kmcurry/status/1574333302022062080"
  img="https://pbs.twimg.com/profile_images/1773236097642123264/nBGahkSO_x96.jpg"
>
needle.tools is a wonderful showcase of what @NeedleTools contributes to 3D via the web. I just love it.
</testimonial>

<testimonial
  name="Brit Gardner"
  role="Senior Software Engineer, Coherence. Ex-Blizzard"
  src="https://x.com/britg/status/1562443905580163072"
  img="https://pbs.twimg.com/profile_images/1360002897048727553/0gscFjvt_x96.jpg"
>
Played with this a bit this morning 🤯🤯 pretty magical
</testimonial>

<testimonial
  name="Marc Wakefield"
  role="Augmented Reality Expert"
  src="https://x.com/mrm_design/status/1567391880169545729"
  img="https://pbs.twimg.com/profile_images/1106863126958493697/tlgb3EyM_x96.jpg"
>
This is huge for WebXR and shared, immersive 3D experiences! Thank you so much for putting in the work on this @NeedleTools crew! Hoping @Apple
 sort out their WebXR situation sooner rather than later. The AR part worked flawlessly on my @SamsungMobile S21.
</testimonial>

<testimonial
  name="Rinesh Thomas"
  role="Game Developer, Assassin's Creed VR"
  src="https://x.com/rineshthomas/status/1566342798063947777?t=z6sG3Z7mol-NfIRfTTKqCQ&s=19"
  img="https://pbs.twimg.com/profile_images/1427818935/rinesh_small_x96.jpg"
>
    This is the best thing I have seen after cinemachine in unity. Unity should acquire this
</testimonial>

<testimonial
  name="Stella Cannefax"
  role="CTO Dora, Game Developer"
  src="https://x.com/0xstella/status/1574853012585172993"
  img="https://pbs.twimg.com/profile_images/1657086025661833237/Yyo56hVY_x96.jpg"
>
Thanks to @NeedleTools, seeing quite a bit of this solution for web-based real time 3d tools - export scenes from Unity, where you can leverage the extensive 3d editor ecosystem & content, and then render them in your own web-based engine
</testimonial>

<testimonial
  name="Pete Patterson" 
  role="Spatial Computing Specialist"
  src="https://x.com/VRSpatialist/status/1572300394285383680"
  img="https://pbs.twimg.com/profile_images/1560370896866533378/aEzq15Cm_x96.jpg"
>
Finally checking out @NeedleTools with Unity. Super easy to get something up and running in the cloud using their
 integrations
</testimonial>

<testimonial 
  name="Dilmer Valecillos" 
  role="Developer Advocate for Mixed Reality @Meta, Tutorial Creator"
  src="https://x.com/Dilmerv/status/1562209049856188420"
  img="https://pbs.twimg.com/profile_images/1876790676610023424/4KHnReTp_x96.jpg"
>
 This is amazing and if you are curious about #WebXR with Unity this will help us get there
</testimonial>

<testimonial 
  name="VRSpatialist" 
  src="https://discord.com/channels/717429793926283276/722046635525537842/1030201907513405530"
  img="https://cdn.discordapp.com/avatars/563549194137239552/b0e0746e19ad79db53cf4465ebb0791e.webp?size=96"
>
 I am a long time Unity dev and recently started playing with Needle Tools and I love it! It's a great on ramp for Unity devs who want to learn WebXR and three.js. The runtime engine is awesome and it was pretty easy to create my own custom component
</testimonial>

<testimonial
  name="Unity for Digital Twins"
  role="Powering the enterprise metaverse"
  src="https://x.com/DigitalTwin/status/1576934958681055233"
  img="https://pbs.twimg.com/profile_images/1659581582657863681/xl4kQKA__x96.jpg"
>
We just gotta say WOW 🤩
</testimonial>

<testimonial
  name="Matthew Pieri" 
  role="Game Developer"
  src="https://discord.com/channels/717429793926283276/1097572505738301571/1097572505738301571"
  img="https://pbs.twimg.com/profile_images/1109900618657878016/3cej9Jql_x96.jpg"
>
Spent the last 2.5 months building this game, never built a game/never used unity before, but absolutely loving the whole process with needle tools. So rapid!  Would love to make a career building AR experiences! 
</testimonial>

<testimonial 
  name="Yuzu" 
  role="Web Game Developer"
  src="https://discord.com/channels/717429793926283276/1264966222467043433/1265268833485066240"
  img="https://cdn.discordapp.com/avatars/559315446105636875/c85b503132ed62d741a37cf9174646d2.webp?size=96"
>
My workflow has been optimized 10X ever since i started using needle
</testimonial>  

</quoteslides>



<actiongroup>
    <action href="getting-started/">
    Get started ⭐
    </action>
    <action href="features-overview">
    Features 🎨
    </action>
    <action href="https://engine.needle.tools/samples?utm_source=needle_docs&utm_content=actionbutton">
    Samples 👓
    </action>
    <action subtitle="with AI support" href="https://forum.needle.tools?utm_source=needle_docs&utm_content=actionbutton">
    Get Help 💬
    </action>
    <action href="https://cloud.needle.tools">
    Cloud ⛅️
    </action>
</actiongroup>



**Needle Engine** is a web engine for complex and simple 3D applications alike. Work on your machine and deploy anywhere. Needle Engine is flexible, extensible and has built-in support for **collaboration and XR**. It is built around the **glTF standard** for 3D assets. 

Powerful integrations for **Unity** and **Blender** allow artists and developers to collaborate and manage web applications inside battle-tested 3d editors. These **Integrations** allow you to use editor features for creating models, authoring materials, animating and sequencing animations, baking lightmaps and more with ease. 

Our powerful **compression and optimization pipeline for the web** makes sure your files are ready, small and load fast.


<!-- <video-embed src="https://www.youtube.com/watch?v=p83q4siNeWo" /> -->
 
 <br/>
 <br/>

<actiongroup>

  
<a class="no-external-link-icon" href="https://www.npmjs.com/package/@needle-tools/engine"><img src="https://img.shields.io/npm/v/@needle-tools/engine?style=flat&colorA=ddd&colorB=ddd"/></a>

<a class="no-external-link-icon" href="https://engine.needle.tools/docs/getting-started/"><img src="https://img.shields.io/npm/dt/@needle-tools/engine.svg?style=flat&colorA=ddd&colorB=ddd"/></a>


<a class="no-external-link-icon" href="https://discord.needle.tools"><img src="https://img.shields.io/discord/717429793926283276?style=flat&colorA=ddd&colorB=ddd&label=discord&logo=discord&logoColor=ffffff"></a>
  

</actiongroup>




<p></p> 
<copyright></copyright>

<ClientOnly>
<removeserviceworker/>
</ClientOnly>

---
title: Additional Modules
---

Projects can be composed of re-usable pieces that we call [**NpmDef**](./project-structure.md#npm-definition-files) (which stands for Npm Defintion File). 

Below you can find links to other repositories that contain Unity packages. These packages can be installed like any Unity package and used in your own projects. They usually contain eihter examples or modules that we use ourselves, but that are not ready to be part of the core Needle Engine.  


   - **Custom Timeline Tracks**  
     Video Track (sync video playback to a Timeline)
     CssTrack (control css properties from the Timeline)
     
   - **Splines (for Unity 2022.1+)**  
     Export splines to three.js
     SplineWalker for controlling camera motion based on a spline
     Timeline track to control SplineWalker

   - **Google Drive Integration**  
     Sketches around drag-drop integration of Google Drive, file picking, app integration

    
[Github repository](https://github.com/needle-tools/needle-engine-modules)

# Networking

Needle Engine includes a full networking solution for multiplayer experiences.  
A shared world state, voice chat, session persistence, and more can be achieved with our networking components and APIs. You can network your own components with a choice of automatic or manual networking.  

Networking in Needle Engine is  based on [Websockets](https://github.com/jjxxs/websocket-ts). Automatic networking uses JSON data for ease of use. For complex usecases and high-performance requirements, we use [Flatbuffers](https://google.github.io/flatbuffers/).

Access to core networking functionality can be obtained by using ``this.context.connection`` from a component. The default backend server connects users to rooms. Users in the same room will share state and receive messages from each other.

## Networking Concepts

### Rooms and State

At the heart of networking in Needle Engine is the concept of synchronized rooms. Each room has an ID, and users connect to a room by providing this ID. Rooms are stored on a server, and users can join and leave rooms at any time.  
When a user joins a room, they receive the current state of the room, apply that current state to their scene, and then listen for changes to the room state.  
When a user leaves a room, they stop listening for changes to the room state. 

Room state is stored as JSON data on the server, so all changes are persistent. This means that room state is not only useful for networking, but also for persisting actions of a single user.

Needle can provide _view-only IDs_ for rooms. When accessing a room with a view-only ID, the user will not be able to interact with the room, but will be able to see the current state and get live updates. This is useful for presentations or demonstrations.

### Ownership

Objects in a room can be _owned_ by a user. This means that only the owner of an object can change its state. 
By default, objects have no owner.
Components like `DragControls` will request ownership of an object before actually moving it.
In custom components, you can control how ownership is handled. 
There may be no ownership required, ownership may be allowed to be transferred to another user automatically, or ownership may be transferred only by a specific action.

When a user leaves a room, objects owned by that user will either be deleted or have ownership reset, depending on how the object was created.

## Enable Networking for your project

1. Add a `SyncedRoom` component to your scene. By default, this will use networking infrastructure provided by Needle.
2. Add a `SyncedTransform` component to a object whose movement you want to synchronize across the network.
3. Add a `DragControls` component to the same object.
4. Run the project. In the browser, click on "Join Room" and copy the URL.
5. Open a new browser window and paste the URL. You should now see the same object in both windows. Try dragging the object in one window and see it move in the other window.

The `DragControls` component, like many other Needle components, has built-in networking support. 
Ownership will be transferred to whoever starts dragging the object. 

## Built-In Components with Networking Support

| Component | Description |
| --- | --- |
| `SyncedRoom` | Handles networking connection and connection to a room. |
| `SyncedTransform` | Handles synchronizing transforms. |
| `SyncedCamera` | Spawns a prefab for any user connected to the room which will follow their position. |
| `VoIP` | Handles voice-over-IP audio connections, microphone access etc. between users. |
| `ScreenCapture` | Handles screensharing via web APIs. |
| `Networking` | Use to customize the server backend URL. Also allows setting a local server for development. |
| `DragControls` | Handles dragging objects. Ownership will automatically be passed to the last user dragging an object. |
| `Duplicatable` | Handles duplicating objects. Duplicated objects are instantiated for everyone in the room. |
| `Deletable` | Handles deleting objects. Deletions are synchronized across the network. |
| `DeleteBox` | Handles deleting objects that have a "Deletable" component when they are dragged into a box volume. |
| `PlayerSync` | Powerful component that instantiates a specific object for each connected player. |
| `PlayerState` | Add this component to objects that are assigned to `PlayerSync`. |
| `PlayerColor` | Simple component for player-specific colors. Each user gets assigned a random color upon joining a room. This component assigns that color to the object's main material. |
| `WebXR` | Handles synchronizing user avatars (hands and heads). |

## Automatic Networking for custom Components

Fields in your own components can be networked very easily. Changes to the field will automatically be detected and sent to all users in the room. The changes are also persisted as part of the Room State, so users that join the room later will receive the current state of the field as well, ensuring everyone sees the same data.

To automatically network a field in a component, decorate it with the ``@syncField()`` decorator:

::::code-group
:::code-group-item Sync a number
```ts twoslash
import { Behaviour, syncField, IPointerClickHandler } from "@needle-tools/engine"

export class SyncedNumber extends Behaviour implements IPointerClickHandler {

    // Use `@syncField` to automatically network a field. 
    // You can optionally assign a method or method name to be called when the value changes.
    @syncField("myValueChanged")
    mySyncedValue?: number = 1;
    
    private myValueChanged() {
       console.log("My value changed", this.mySyncedValue);
    }
    
    onPointerClick() {
       this.mySyncedValue = Math.random();
    }
}
```
:::
:::code-group-item Sync an object's color
<!-- SAMPLE network color change -->
:::
::::

Note that syncField has an optional parameter to specify a method that should be called when the value changes. This method should be defined in the same class. 

::: tip Custom Project Setup
If you're using a custom project setup, you need to have ``experimentalDecorators: true`` in your ``tsconfig.json`` file for syncField decorators to work. Projects created with Needle Starters have this enabled by default.
:::

## Creating and destroying objects

Often, you want to create and destroy objects at runtime, and of course these changes should be synchronized across the network. 

The `PlayerSync` component simplifies this process by automatically instantiating a specific object for each connected player. 
When a player leaves the room, the object is destroyed for all users.

Additionally, Needle Engine provides two high-level methods:
- [`syncInstantiate()`](https://engine.needle.tools/docs/api/latest/syncInstantiate) to duplicate objects across the network. 
- [`syncDestroy()`](https://engine.needle.tools/docs/api/latest/syncDestroy) to destroy objects across the network.

> 🏗️ Code Samples Under Construction

## Manual Networking

Needle Engine also provides a low-level API for sending and receiving messages. We call this "manual networking". The principles are the same, but you're in full control for sending and receiving messages and how to handle them.

### Sending Messages

Send a message to all users in the same room: 
```ts
this.context.connection.send(key: string, data: IModel | object | boolean | string | number | null);
```

### Receiving Messages

You can subscribe to events in the room using a specific key.
Typically, you want to match this with unsubscribing:   

- subscribe in `onEnable` and unsubscribe in `onDisable`  
  With this approach, no messages will be received while the object is disabled.

- or subscribe in `start` and unsubscribe in `onDestroy`  
  With this approach, messages will still be received while the object is disabled.

```ts
this.context.connection.beginListen(key:string, callback:(data) => void)
```   

Unsubscribe from events: 
```ts
this.context.connection.stopListen(key:string)
```

### Controlling message persistence

When sending network messages, the low-level API allows you to decide whether that message should be persistet (saved in the room state) or not (only sent to users currently in the room). To persist a message, make sure it has a `guid` field. This field is typically used to apply the message data to a specific object, by providing that object's guid. If you want target a specific object (and thus,  include a `guid` field) but want the data to not be persisted, set the `dontSave` field to `true` in your message.

All persistent messages are saved in the room state and will be sent to users that connect at a later point. Non-persistent messages are only sent to users currently in the room, which is useful for effects (like playing a sound effect) that don't make sense to play for users that are currently not in the room. Optionally, you can include a `deleteOnDisconnect` field in your message to delete this particular message when the user disconnects.

```ts
// This message will be sent to all users currently in the room,
// AND to users that join the room later.
this.context.connection.send("my-message", { guid: this.guid, myData: "myValue" });

// This message will be sent to all users currently in the room,
// but NOT be sent to users that join the room later.
this.context.connection.send("my-message", { guid: this.guid, myData: "myValue", dontSave: true });

// This message will be sent to all users currently in the room,
// but NOT be sent to users that join the room later.
this.context.connection.send("my-message", { myData: "myValue" });

// This message will be sent to all users currently in the room,
// AND to users that join the room later,
// but will be deleted from the room state when the user disconnects.
this.context.connection.send("my-message", { guid: this.guid, myData: "myValue", deleteOnDisconnect: true });
```

To delete state for a specific guid from the backend storage, set the message key to `delete-state` and target a specific object with its guid: `{ guid: "guid_to_delete" } `.

```ts
this.context.connection.send("delete-state", { guid: "guid_to_delete" });
```

### Using debug flags to understand network messages

There are several debug flags that can be used to dive deeper into network messages. 
These can be appended the the page URL, like `https://localhost:3000/?debugnet`.

| Flag | Description |
|------|-------------|
| `?debugnet` | Log all incoming and outgoing network messages to the console |
| `?debugowner` | Log all ownership requests and changes to the console |
| `?debugnetbin` | Log additional information for incoming and outgoing binary messages |


## Networking Lifecycle Events

The following events are available to listen to in your components. They describe common network events that you might want to react to in your components, like yourself or another user joining or leaving a room. 

```ts
// Listen to the event when *you* have joined a networked room
this.context.beginListen(RoomEvents.JoinedRoom, ({room, viewId, allowEditing, inRoom}) => { ... });

// Listen to the event when *you* have left a networked room
this.context.beginListen(RoomEvents.LeftRoom, ({room}) => { ... });

// Listen to the event when *another user* has joined your networked room
this.context.beginListen(RoomEvents.UserJoinedRoom, ({userId}) => { ... });

// Listen to the event when *another user* has left your networked room
this.context.beginListen(RoomEvents.UserLeftRoom, ({userId}) => { ... });

// This event is received after all current room state has been sent to the client
this.context.beginListen(RoomEvents.RoomStateSent, () => { ... });
```

- [See all Room Events in the API docs](https://engine.needle.tools/docs/api/latest/RoomEvents)
- [See all Ownership Events in the API docs](https://engine.needle.tools/docs/api/latest/OwnershipEvent)
- [See all Connection Events in the API docs](https://engine.needle.tools/docs/api/latest/ConnectionEvents)

## Using Needle Networking Servers

By default, networked Needle scenes connect to cloud infrastructure managed and provided by Needle. There is no additional setup needed, and currently no additional cost for using this service.  

Typically, this will work fine for around 15-20 users in the same room. Once your project matures, you can upgrade to a bigger/better/stronger networking solution, by hosting your own networking server. 

## Hosting your own Networking Server

You might want to host your own networking server for larger deployments or to have more control over the networking infrastructure and implementation. 

Our networking server is available on NPM [own networking package](https://fwd.needle.tools/needle-engine/packages/needle-engine-networking) as node.js package. The package contains pre-configured integrations for the popular web frameworks [Fastify](https://www.npmjs.com/package/fastify) and [Express](https://www.npmjs.com/package/express), and can be integrated into other Node.js server frameworks as well.

::: tip For quick experiments: Remix on Glitch
You can remix a simple networking server running on Glitch from this page: [needle-networking.glitch.me](https://needle-networking.glitch.me/) by clicking the button in the bottom right corner.

The default Glitch server instance is small and can only handle a limited amount of users. If you expect more than 15-20 people to be in your scene at the same time, you should consider hosting your networking server elsewhere (like on Google Cloud or AWS).
:::

::::code-group
:::code-group-item Fastify
```js
import networking from "@needle-tools/needle-networking";
networking.startServerFastify(fastifyApp, { endpoint: "/socket" });
```
:::
:::code-group-item Express
```js
import networking from "@needle-tools/needle-networking";
networking.startServerExpress(expressApp, { endpoint: "/socket" });
```
:::
:::code-group-item Custom Integration
```js
import { init, onConnection } from "@needle-tools/networking";

// Add your framework-specific websocket implementation here. 
// You can view the fastify and express implementations in server.js for reference.
class WebsocketConnector {
    constructor(frameworkWebsocket) {
        // Your implementation.
    }
    on(event, callback) {
        // Your implementation. When receiving a message in the websocket connection, call the callback.
        // 'event' can be 'message' or 'close'.
    }
    send(key, value) {
        // Your implementation. Pass the message along to the websocket connection.
    }
}
const options = { endpoint: "/socket" };
init(options);
yourFramework.createWebsocketRoute(options.endpoint, frameworkWebsocket => {
    onConnection(new WebsocketConnector(frameworkWebsocket));
});
```
:::
::::

::: tip Example on Glitch.com
See the code on [glitch.com/edit/#!/needle-networking](https://glitch.com/edit/#!/needle-networking?path=server.js) for an example of how to integrate Needle Networking with an Express server.
:::

### Configuration

The following options are available:

| Option | Description |
| -- | -- |
| `options.endpoint` *string* | Optional. Relative server endpoint. For example, `/socket` will start the websocket endpoint on `yourserver/socket`. Defaults to `/`. |
| `options.maxUsers` *number* | Maximum number of concurrent users on a server. Defaults to `50`. |
| `options.defaultUserTimeout` *number* | Time in seconds after which a user is considered disconnected. Defaults to `30`. |
| `process.env.VIEW_ONLY_SALT` *string* | Salt value used for generating view-only room IDs from regular room IDs. Defaults to a predefined salt value. |
| `process.env.NEEDLE_NETWORKING_S3_*` *string* | Enable S3 storage. See below for the full list of environment variables you need to set for this. When not set, the default storage is used (JSON files on disk). |

The networking server will automatically manage connecting and disconnecting users, receiving and sending messages, and persisting room state.

Custom networking servers can be deployed anywhere, for example on Google Cloud. For further instructions please refer to this repository: [Local Needle Networking Server](https://fwd.needle.tools/needle-engine/local-networking-repository)

::: tip Different server locations for local and hosted development
If you're working on custom networking code, you may want to use different server locations for local development and the hosted app. You can set individual server URLs in the `Networking` component: 

![Needle Engine Networking component with networking server hosted elswhere](/imgs/networking_absolute.webp)
::: 

#### State Storage

Network state is by default stored to disk on the server as JSON files in the `/.data` directory.  
Each room has its own file, and the state is sent to connecting clients when they join a room.

Optionally, the networking state can be stored with an S3 compatible storage provider. Use the following environment variables to enable S3 storage:  

```bash
NEEDLE_NETWORKING_S3_ENDPOINT=
NEEDLE_NETWORKING_S3_REGION=
NEEDLE_NETWORKING_S3_BUCKET=
NEEDLE_NETWORKING_S3_ACCESS_KEY_ID=
NEEDLE_NETWORKING_S3_ACCESS_KEY=
NEEDLE_NETWORKING_S3_PREFIX= # all state saved in the bucket will be prefixed with this string. This can be a path e.g. `my_state/` or a unique id `server_123_`
```

## Local Networking Server

For testing and development purposes, you can run the Needle Engine networking package on a local server. We have prepared a repository that is set up to host the websocket package and to make that easy for you. 

1. Download the local server sample from [github.com/needle-tools/networking-local](https://fwd.needle.tools/needle-engine/local-networking-repository)  
2. Follow the instructions in the README to set up the server. The server will run on `wss://localhost:9001/socket` by default.
3. Add the `Networking` component to your scene.
4. Paste the local server address into the `Localhost` field on the `Networking` component.

## Advanced: Customizing WebRTC settings for peer.js

Needle Engine `Screencapture` (Screensharing) and `VoIP` (Voice communication) components use [peer.js](https://peerjs.com/) for networking audio and video. Peer.js uses WebRTC under the hood.

Needle Engine uses reasonable defaults for peerjs. If you want to modify those defaults, you can call 
```ts
setPeerOptions(opts: PeerjsOptions);
```
with your custom settings. This can be used to modify the hosting provider for ICE/STUN/TURN servers, for example when you use your own WebRTC servers.

## Advanced: Server and Client Message Formats

::: warning For informational purposes. Use the APIs provided by Needle Engine instead.
Typically, you do not need to interact with these message formats directly, as the low-level networking API already handles parsing messages and giving you the correct types. The information here is provided for advanced users who want to understand the underlying message formats or implement their own networking solutions.
:::

Messages are sent in JSON format. They always have a `key` field that describes the type of message, and a `data` field that contains the message payload. The `data` field can be any JSON-serializable object.

### Built-in Room Events

::::code-group
:::code-group-item Join
```json
// Sent to the server to attempt joining a room.
{
    "key": "join-room",
    "data": {
        "room": string,
        "viewOnly": boolean,
    }
}
```
:::
:::code-group-item Leave
```json
// Sent to the server to leave a room.
{
    "key": "leave-room",
    "data": {
        "room": string
    }
}
```
:::
:::code-group-item JoinedRoom
```json
// Sent to the client when the local user has joined a room.
// Type: JoinedRoomResponse
{
    "key": "joined-room",
    "room": string,
    "viewId": string,
    "allowEditing": boolean,
    "inRoom": string[] // connection IDs
}
```
:::
:::code-group-item LeftRoom
```json
// Sent to the client when the local user has left a room.
// Type: LeftRoomResponse
{
    "key": "left-room",
    "room": string
}
```
:::
:::code-group-item UserJoinedRoom
```json
// Sent to the client when any user has joined a room.
// Type: UserJoinedOrLeftRoomModel
{
    "key": "user-joined-room",
    "data": {
        "userId": string // connection ID
    }
}
```
:::
:::code-group-item UserLeftRoom
```json
// Sent to the client when any user has left a room.
// Type: UserJoinedOrLeftRoomModel
{
    "key": "user-left-room",
    "data": {
        "userId": string // connection ID
    }
}
````
:::
:::code-group-item RoomStateSent
```json
// Sent to the client after the complete room state has been sent.
{
    "key": "room-state-sent",
    "room": string // room name 
}
```
:::
::::

### Built-In Utility Events

::::code-group
:::code-group-item ConnectionInfo
```json
// Sent to the client when the connection is established.
{
    "key": "connection-start-info",
    "data": {
        "id": string // connection ID
    }
}
```
:::
:::code-group-item syncInstantiate
```json
// Used by the syncInstantiate() API to create a new instance of an asset.
// Type: NewInstanceModel
{
    "key": "new-instance-created",
    "data": {
        "guid": string,
        "originalGuid": string,
        "seed": number | undefined,
        "visible": boolean | undefined,
        "dontSave": boolean | undefined,

        "parent": string | undefined,
        "position": { x: number, y: number, z: number } | undefined,
        "rotation": { x: number, y: number, z: number, w: number } | undefined,
        "scale": { x: number, y: number, z: number } | undefined,

        "deleteStateOnDisconnect": boolean | undefined
    }
```
:::
:::code-group-item syncDestroy
```json
// Used by the syncDestroy() API to destroy an instance of an asset.
// Type: DestroyInstanceModel
{
    "key": "instance-destroyed",
    "data": {
        "guid": string,
        "dontSave": boolean | undefined
    }
}
```
:::
:::code-group-item Ping
```json
// Sent to the server every few seconds to keep the connection alive.
{
    "key": "ping",
    "data": {}
}
```
:::
:::code-group-item Pong
```json
// Sent by the server in response to a ping.
{
    "key": "pong",
    "data": {}
}
```
:::
:::code-group-item DeleteState
```json
// Sent to the server to delete state for a specific guid.
{
    "key": "delete-state",
    "data": {
        "guid": <string>
    }
}
```
:::
:::code-group-item DeleteAllState
```json
// Sent to the server to delete ALL current room state.
{
    "key": "delete-all-state",
    "data": {}
}
```
::::

### Built-In Ownership Events

::::code-group
:::code-group-item OwnershipRequest
```json
{
    "key": 
      "request-has-owner" | 
      "request-ownership" | 
      "remove-ownership",
    "data": {
        "guid": string
    }
}
```
:::
:::code-group-item OwnershipResponse
// Type: OwnershipResponse
```json
{
    "key": 
      "response-has-owner",
    "data": {
        "guid": string,
        "value": boolean
    }
}
```
:::
::: code-group-item OwnershipBroadcastResponse
```json
{
    "key": 
      "gained-ownership" | 
      "lost-ownership" | 
      "gained-ownership-broadcast" | 
      "lost-ownership-broadcast",
    "data": {
        "guid": string,
        "owner": string
    }
}
```
:::
::::

### Built-In Flatbuffer Schemas

Flatbuffer messages are sent directly as binary messages.

::::code-group
:::code-group-item SyncedTransform ('STRS')
```cs
<!-- SAMPLE node_modules/@needle-tools/engine/src/engine-schemes/transforms.fbs -->
```
:::
:::code-group-item SyncedCamera ('SCAM')
```cs
<!-- SAMPLE node_modules/@needle-tools/engine/src/engine-schemes/syncedCamera.fbs -->
```
:::
:::code-group-item Vec2|3|4
```cs
<!-- SAMPLE node_modules/@needle-tools/engine/src/engine-schemes/vec.fbs -->
```
:::
::::

## Advanced: Binary Messages in the Flatbuffer format

JSON messages are easy to use and understand, but are typically larger in memory and bandwidth. For large amounts of data, or when sending fast updates, binary messages are faster and more efficient. You can use Flatbuffers messages in Needle Engine for cases where that is required. Using Flatbuffers requires additional setup steps like defining and compiling a message schema, and is harder to debug since you're dealing with binary messages.

Note that when sending and receiving flatbuffer messages, there is no `key` field – the message type is part of the Flatbuffer schema. What you send and receive over the Websocket connection is a single binary buffer.  

Send a binary message to all users in the same room:
```ts
this.context.connection.sendBinary(byteArray: Uint8Array);
```

Subscribe to binary messages in Flatbuffer format: 
```ts
this.context.connection.beginListenBinary(identifier:string, callback:(data : ByteBuffer) => void);
```

Unsubscribe from binary messages:
```ts
this.context.connection.stopListenBinary(identifier:string);
```

#### Flatbuffers Sample Code

Before you can send and receive Flatbuffer messages, you need to define a schema and compile it to TypeScript. Then, register the schema with the networking system and use the generated schema methods to create and parse messages.

- [Built-in Flatbuffer schemas in Needle Engine](#built-in-flatbuffer-schemas)
- [Generating a schema](https://google.github.io/flatbuffers/flatbuffers_guide_writing_schema.html)
- [Using the schema compiler](https://google.github.io/flatbuffers/flatbuffers_guide_using_schema_compiler.html)
- [Flatbuffers in Typescript](https://google.github.io/flatbuffers/flatbuffers_guide_use_typescript.html)


::::code-group
:::code-group-item Register a schema
```ts
// Register a new Flatbuffer schema with the networking system
import { registerBinaryType } from '@needle-tools/engine';
import { MyDataModel } from 'my-data-model.js';

const MySchemaIdentifier = "MYSC";

registerBinaryType(MySchemaIdentifier, MyDataModel.getRootAsSyncedTransformModel);
```
:::
:::code-group-item Send Messages
```ts
// Prepare data for sending by creating a Flatbuffer message:
import { MyDataModel } from 'my-data-model.js';

const MySchemaIdentifier = "MYSC";
const builder = new flatbuffers.Builder();

// Construct a Flatbuffer message
function createMyCustomModel(somePayload: string): Uint8Array {
    builder.clear();
    MyDataModel.startMyDataModel(builder);    
    const guidObj = builder.createString(guid);
    MyDataModel.addSomePayload(builder, guidObj);
    const res = MyDataModel.endMyDataModel(builder);
    builder.finish(res, MySchemaIdentifier);
    return builder.asUint8Array();
}

// Send the data
function sendData() {
    const data = createMyCustomModel("your-payload", this, true);
    this.context.connection.sendBinary(data);
}
```
:::
:::code-group-item Receive Messages
```ts
// Subscribe to receive this specific message type:
import { MyDataModel } from 'my-data-model.js';

const MySchemaIdentifier = "MYSC";

this.context.connection.beginListenBinary(MySchemaIdentifier, (data) => {
    const model = MyDataModel.getRootAsMyDataModel(data);
    console.log("Received binary message", model, model.somePayload());
});
```
:::
::::

::: tip Custom Flatbuffer messages and persistence
Currently, custom binary messages can't be persisted on the networking server. Modify the networking server and add your custom flatbuffer schemas to ensure the guid property can be processed.
:::

## Summary

Needle Engine makes the complex topic of networking approachable and easy to use. You can get started with automatic networking for your components with just a few lines of code, and you can dive deeper into manual networking when you need more control.

---
title: Web Project Structure
---

# Needle Engine Project Structure

### Web Project Files

| | |
| --- | --- |
| **Needle Engine** | |
| [`needle.config.json`](./reference/needle-config-json.md) | Configuration for Needle Engine builds and integrations |
| **Ecosystem** | |
| `package.json` | Project configuration containing name, version, dependencies and scripts |
| `tsconfig.json` | Typescript compiler configuration |
| `.gitignore` | Files and folders to be ignored in git |
| `vite.config.js` | Contains vite specific configuration.<br/>It also adds the Needle Engine vite plugins. |


### Default Vite project structure

Our main project template uses the superfast [vite](https://vitejs.dev/) bundler. The following shows the structure of the Vite template that we created and ship (altough it is possible to adapt it to your own needs).

| | |
| --- | --- |
| **Folders** | |
| `assets/` | The asset folder contains exported assets from Unity. E.g. generated ``gltf`` files, audio or video files. It is not recommended to manually add files to ``assets`` as it will get cleared on building the distribution for the project.
| `include/` | (optional) - If you have custom assets that you need to reference/load add them to the include directory. On build this directory will be copied to the output folder.
| `src/generated/` | The generated javascript code. Do not edit manually!
| `src/scripts/` | Your project specific scripts / components
| `src/styles/` | Stylesheets
| `*` | You can add any new folders here as you please. Make sure to [copy](./reference/needle-config-json.md) them to the output directory when building |
| **Files** | |
| `index.html` | The landing- or homepage of your website
| `vite.config` | The [vite config](https://vitejs.dev/config/). Settings for building the distribution and hosting the development server are made here. It is usually not necessary to edit these settings.
| `src/main.ts` | Included from `index.html` and importing `needle-engine`
| `*` | You can add any new files here as you please. Make sure to [copy](./reference/needle-config-json.md) them to the output directory when building (unless they are just being used during development) |

Our exporter can be used with other project structures as well, vite is just our go-to frontend bundling tool because of its speed. Feel free to set up your JavaScript project as you like. 

[Learn more in the docs about bundling and usage with other frameworks](html.md)



---

#### Continue Reading

- [Typescript Guide for Unity Developers](./getting-started/for-unity-developers.md)
- [Typescript Essentials](./getting-started/typescript-essentials.md)
- [Writing custom scripts](./scripting.md)
- [Everywhere Actions](./everywhere-actions.md)

---
title: Samples Projects
---

<sample src="https://engine.needle.tools/samples-uploads/physics-cannon/" />

## Samples to download and play
View all samples at [engine.needle.tools/samples](https://engine.needle.tools/samples) with a live preview and links for download and installation.
  
<!-- - [Brune Simons 20k](https://engine.needle.tools/samples/buno-simon-20k)  
  *Physics*
- [Stencil Portals](https://engine.needle.tools/samples/stencil-portals)  
  *URP RenderObject stencil to threejs*
- [Custom Shaders](https://engine.needle.tools/samples/custom-shaders)  
  *Unity ShaderGraph to threejs*
- [Sandbox](https://needle-tiny-starter.glitch.me/)  
  *Multi-user sandbox builder*
- [VideoPlayer](https://needle-videoplayer-sample.glitch.me/)  
  *Minimal Unity VideoPlayer sample*  
- [UI Button](https://engine.needle.tools/samples/ui-button)   
  *Minimal UI button event sample*
- [Spatial Trigger](https://needle-spatial-trigger-sample.glitch.me/)  
  *Define areas in your world and hookup events from Unity*
- and [more](https://engine.needle.tools/samples/)

---

<sample src="https://engine.needle.tools/samples/particles/" />

<video-embed src="https://user-images.githubusercontent.com/5083203/189970610-9f913d5e-4f77-4a64-bc15-beb8220e82cf.mp4" />
 
<video-embed src="https://user-images.githubusercontent.com/5083203/189973474-2d907e1c-fae3-439f-a349-5fe2c9b25b15.mp4" />
 
<video-embed src="https://user-images.githubusercontent.com/5083203/190458268-c66bad1e-fd68-4171-85b0-2ce001591650.mp4" />

<sample src="https://engine.needle.tools/samples/stencil-portals/" />

<sample src="https://engine.needle.tools/samples/iframe-content/" /> -->


<!-- <actiongroup>
<action href="https://github.com/needle-tools/needle-engine-samples">Get the samples on github</action>
</actiongroup> -->

---
title: Scripting Examples
description: A collection of useful script snippets and examples.
---

# Scripting Examples

If you are new to scripting we **highly recommend** reading the following guides first:

- [Beginner Guide: Typescript Essentials](./getting-started/typescript-essentials.md)
- [Beginner Guide: Needle Engine for Unity Developers](./getting-started/for-unity-developers.md)
- [Video tutorial: How to write custom components](https://youtu.be/uf5UK0bLHlY?si=82U_2L4n2V7XL7RJ)

Below you will find a few basic scripts as a quick reference. 

We also offer a lot of sample scenes and complete projects that you can download and use as a starting point:  
- [Visit Samples Website](https://engine.needle.tools/samples?utm_source=needle_docs&utm_content=scripting_examples)
- [Download Samples Package](https://engine.needle.tools/downloads/unity/samples)
- [Needle Engine Stackblitz Collection](https://stackblitz.com/@marwie/collections/needle-engine)
- [Needle Engine API](https://engine.needle.tools/api)

## Basic component 
<stackblitz file="@code/basic-component.ts"></stackblitz> 
@[code ts twoslash](@code/basic-component.ts)

see [scripting](scripting#lifecycle-methods) for all component events

## Reference an Object from Unity
@[code ts twoslash](@code/component-object-reference.ts)  

## Reference and load an asset from Unity (Prefab or SceneAsset)
@[code ts twoslash](@code/component-prefab.ts)

## Reference and load scenes from Unity
::: tip
Find a [working example in our samples](https://engine.needle.tools/samples/multi-scenes-(dynamic-loading)) to download and try
:::

@[code ts twoslash](@code/component-scene.ts)

## Receive Clicks on Objects
Add this script to any object in your scene that you want to be clickable. Make sure to also have an `ObjectRaycaster` component in the parent hierarchy of that object.  

<stackblitz file="@code/component-click.ts">
test
</stackblitz> 

@[code ts twoslash](@code/component-click.ts)


## Networking Clicks on Objects

Add this script to any object in your scene that you want to be clickable. Make sure to also have an `ObjectRaycaster` component in the parent hierarchy of that object.   
The component will send the received click to all connected clients and will raise an event that you can then react to in your app. If you are using Unity or Blender you can simply assign functions to call to the `onClick` event to e.g. play an animation or hide objects.

@[code ts twoslash](@code/component-click-networking.ts)

### Play Animation on click
@[code ts twoslash](@code/component-animation-onclick.ts)

## Reference an Animation Clip
This can be useful if you want to run your custom animation logic.   
You can also export an array of clips.
@[code ts twoslash](@code/component-animationclip.ts)


## Create and invoke a UnityEvent

@[code ts twoslash](@code/component-unityevent.ts)
::: tip
EventList events are also invoked on the component level. This means you can also subscribe to the event declared above using ``myComponent.addEventListener("my-event", evt => {...})`` as well.   
This is an experimental feature. Please provide feedback in our [forum](https://forum.needle.tools/?utm_source=needle_docs&utm_content=content)
:::


### Declare a custom event type
This is useful for when you want to expose an event to Unity or Blender with some custom arguments (like a string)
@[code ts twoslash](@code/component-customevent.ts)

_Example use:_  
![20221128-210735_Unity-needle](https://user-images.githubusercontent.com/2693840/204370950-4c89b877-90d7-4e6f-8266-3352e6da16f4.png)

## Use nested objects and serialization

You can nest objects and their data. With properly matching `@serializable(SomeType)` decorators, the data will be serialized and deserialized into the correct types automatically.  

In your typescript component:  
@[code ts twoslash](@code/component-nested-serialization.ts)

In C# in any script:  
@[code](@code/component-nested-serialization-cs.cs)

::: tip
Without the correct type decorators, you will still get the data, but just as a plain object. This is useful when you're porting components, as you'll have access to all data and can add types as required.
:::

## Use Web APIs
::: tip
Keep in mind that you still have access to all web apis and [npm](https://npmjs.org) packages!    
That's the beauty of Needle Engine if we're allowed to say this here 😊
:::

### Display current location
@[code ts twoslash](@code/component-location.ts) 

### Display current time using a Coroutine
@[code ts twoslash](@code/component-time.ts) 

<video-embed src="./videos/component-time.mp4" limit_height />


## Change custom shader property

Assuming you have a custom shader with a property name `_Speed` that is a float value this is how you would change it from a script.   
You can find a live [example to download in our samples](https://engine.needle.tools/samples/shaders/)

<!-- SAMPLE modify custom shader material property -->


## Switching src attribute

See [live example](https://stackblitz.com/edit/needle-engine-cycle-src?file=index.html) on StackBlitz


## Adding new postprocessing effects

Make sure to install [`npm i postprocessing`](https://github.com/pmndrs/postprocessing) in your web project. Then you can add new effects by deriving from `PostProcessingEffect`.  

To use the effect add it to the same object as your `Volume` component.

Here is an example that wraps the [Outline postprocessing effect](https://pmndrs.github.io/postprocessing/public/demo/#outline). You can expose variables and settings as usual as any effect is also just a component in your three.js scene.

@[code](@code/custom-post-effect.ts) 


## Custom ParticleSystem Behaviour


@[code ts twoslash](@code/custom-particle-system-behaviour.ts) 


## Custom 2D Audio Component

This is an example how you could create your own audio component.   
For most usecases however you can use the core AudioSource component and don't have to write code.

@[code ts twoslash](@code/component-2d-audio.ts)


## Arbitrary external files

Use the FileReference type to load external files (e.g. a json file)
@[code ts twoslash](@code/component-filereference.ts)

<!-- SAMPLE receive click from HTML button
## Receiving html element click in component
-->



<!-- SAMPLE disable environment light 
## Disable environment light
-->


<!-- SAMPLE using mediapipe with hands 
## Use mediapipe package to control the 3D scene with hands
Make sure to install the mediapipe package. Visit the github link below to see the complete project setup.  
Try it [live here](https://engine.needle.tools/samples/mediapipe-hands/) - requires a webcam/camera
-->


<!-- SAMPLE Change Color On Collision
## Change Color On Collision
-->

<!-- SAMPLE Physics Trigger Relay
## Physics Trigger Relay
Invoke events using an objects physics trigger methods 
-->

<!-- SAMPLE Auto Reset
## Auto Reset
Reset an object's position automatically when it's leaving a physics trigger
-->

<!-- SAMPLE Play Audio On Collision
## Play Audio On Collision
-->

<!-- SAMPLE Set Random Color
## Set Random Color
Randomize the color of an object on start. Note that the materials are cloned in the `start` method
-->

<!-- SAMPLE Timed Spawn
## Spawn Objects Over Time
-->

---
title: Creating and using Components
tags:
    - scripting
    - serialization
    - csharp
    - typescript
    - javascript
    - components
---

# Creating custom components

If you are new to scripting we **highly recommend** reading the following guides first:

- [Typescript Essentials](./getting-started/typescript-essentials.md)
- [Needle Engine for Unity Developers](./getting-started/for-unity-developers.md)

If you know what you're doing, feel free to jump right into the [Needle Engine API documentation](https://engine.needle.tools/docs/api/latest).

---

Runtime code for Needle Engine is written in [TypeScript](https://typescriptlang.org) (recommended) or [JavaScript](https://javascript.info/). We automatically generate C# stub components out of that, which you can add to GameObjects in the editor. The C# components and their data are recreated by the runtime as JavaScript components with the same data and attached to three.js objects.  

Both custom components as well as built-in Unity components can be mapped to JavaScript components in this way. For example, mappings for many built-in components related to animation, rendering or physics are already [included in Needle Engine](./component-reference.md#unity-components).  

If you want to code-along with the following examples without having to install anything you just click the following link:

- [Create virtual workspace to code along](https://stackblitz.com/fork/github/needle-engine/vite-template?file=src%2Fmain.ts).

----

Our web runtime engine adopts a component model similar to Unity and thus provides a lot of functionality that will feel familiar.
Components attached to three's Object3D objects have lifecycle methods like ``awake``, ``start``, ``onEnable``, ``onDisable``, ``update`` and ``lateUpdate`` that you can implement. You can also use [Coroutines](#coroutines).   

----

## When you don't need to write code

Often, interactive scenes can be realized using Events in Unity and calling methods on built-in components. A typical example is playing an animation on button click - you create a button, add a Click event in the inspector, and have that call Animator.SetTrigger or similar to play a specific animation.  

Needle Engine translates Unity Events into JavaScript method calls, which makes this a very fast and flexible workflow - set up your events as usual and when they're called they'll work the same as in Unity.  

![image](https://user-images.githubusercontent.com/2693840/187314594-7e34905d-e704-4fa3-835c-6b40f11e1c62.png)   
_An example of a Button Click Event that is working out-of-the-box in Needle Engine — no code needed._ 


## Creating a new component
Scripts are written in TypeScript (recommended) or JavaScript.   
There are two ways to add custom scripts to your project:

- Simply add a file with an `.ts` or `.js` extension inside `src/scripts/` in your generated project directory, for example `src/scripts/MyFirstScript.ts`

- Unity specific:   
  Organize your code into NPM Definition Files (npm packages). These help you to modularize and re-use code between projects and if you are familiar with web development they are in fact regular npm packages that are installed locally.  
  In Unity you can create NpmDef files via `Create > NPM Definition` and then add TypeScript files by right-clicking an NpmDef file and selecting `Create > TypeScript`. Please see [this chapter](./project-structure.md#npm-definition-files) for more information.  


In both approaches, source directories are watched for changes and C# stub components or Blender panels are regenerated whenever a change is detected.   
Changes to the source files also result in a hot reload of the running website – you don't have to wait for Unity to recompile the C# components. This makes iterating on code pretty much instant.  

You can even have multiple component types inside one file (e.g. you can declare `export class MyComponent1` and `export class MyOtherComponent` in the same Typescript file).

If you are new to writing Javascript or Typescript we recommend reading the [Typescript Essentials Guide](./getting-started/typescript-essentials.md) guide first before continuing with this guide.

:::details Example: Creating a Component that rotates an object

- **Create a component that rotates an object**  
  Create ``src/scripts/Rotate.ts`` and add the following code:  
```ts twoslash
import { Behaviour, serializable } from "@needle-tools/engine";

export class Rotate extends Behaviour
{
    @serializable()
    speed : number = 1;

    start(){
        // logging this is useful for debugging in the browser. 
        // You can open the developer console (F12) to see what data your component contains
        console.log(this);
    }

    // update will be called every frame
    update(){
        this.gameObject.rotateY(this.context.time.deltaTime * this.speed);
    }
}
```

Now inside Unity a new script called ``Rotate.cs`` will be automatically generated. Add the new Unity component to a Cube and save the scene.   
The cube is now rotating inside the browser.   
Open the chrome developer console by `F12` to inspect the log from the ``Rotate.start`` method. This is a helpful practice to learn and debug what fields are exported and currently assigned. In general all public and serializable fields and all public properties are exported.  

Now add a new field ``public float speed = 5`` to your Unity component and save it. The Rotate component inspector now shows a ``speed`` field that you can edit. Save the scene (or click the ``Build`` button) and note that the javascript component now has the exported ``speed`` value assigned. 
:::

:::details Create component with a custom function
Refer to the [Typescript Essentials Guide](./getting-started/typescript-essentials.md) to learn more about the syntax and language.
```ts twoslash
import { Behaviour } from "@needle-tools/engine";

export class PrintNumberComponent extends Behaviour
{
    start(){
      this.printNumber(42);
    }
    
    private printNumber(myNumber : number){
        console.log("My Number is: " + myNumber);
    }
}
```
:::

:::details Version Control & Unity
While generated C# components use the type name to produce stable GUIDs, we recommend checking in generated components in version control as a good practice. 
::: 


## Component architecture
Components are added to three.js `Object3Ds`. This is similar to how Components in Unity are added to `GameObjects`. Therefore when we want to access a three.js Object3D, we can access it as ``this.gameObject`` which returns the `Object3D` that the component is attached to.  

***Note**: Setting ``visible`` to false on a Object3D will act like ``SetActive(false)`` in Unity - meaning it will also disable all the current components on this object and its children. Update events for inactive components are not being called until ``visible`` is set to true again.* If you want to hide an object without affecting components you can just disable the Needle Engine `Renderer` component.

### Lifecycle methods

Note that lifecycle methods are only being called when they are declared. So only declare `update` lifecycle methods when they are actually necessary, otherwise it may hurt performance if you have many components with update loops that do nothing.

| Method name | Description |
| -- | --
| `awake()` | First method being called when a new component is created
| `onEnable()` | Called when a component is enabled (e.g. when ``enabled`` changes from false to true)
| `onDisable()` | Called when a component is disabled (e.g. when ``enabled`` changes from true to false)
| `onDestroy()` | called when the Object3D or component is being destroyed
| `start()` | Called on the start of the first frame after the component was created
| `earlyUpdate()` | First update event
| `update()` | Default update event
| `lateUpdate()` | Called after update
| `onBeforeRender()` | Last update event before render call
| `onAfterRender()` | Called after render event

### Physic event methods
| Method name | Description |
| -- | --
| `onCollisionEnter(col : Collision)` | 
| `onCollisionStay(col : Collision)` | 
| `onCollisionExit(col : Collision)` | 
| `onTriggerEnter(col : Collision)` | 
| `onTriggerStay(col : Collision)` | 
| `onTriggerExit(col : Collision)` | 

### Input event methods
| Method name | Description |
| -- | --
| `onPointerEnter(args : PointerEventData)` | Called when a cursor starts to hover over an object (or any of it's children)
| `onPointerMove(args : PointerEventData)` | Called when a cursor moves over an object (or any of it's children)
| `onPointerExit(args : PointerEventData)` | Called when a cursor exists (stops hovering) an object
| `onPointerDown(args : PointerEventData)` | Called when a cursor is pressed over an object 
| `onPointerUp(args : PointerEventData)` | Called when a cursor is released over an object
| `onPointerClick(args : PointerEventData)` | Called when a cursor is clicked over an object


### XR event methods
*requires Needle Engine >= 3.32.0*
| Method name | Description |
| -- | --
| `supportsXR(mode: XRSessionMode)` | Optionally implement if you only want to receive XR callbacks for specific XR modes like `immersive-vr` or `immersive-ar`. Return `true` to notify the system that you want callbacks for the passed in mode
| `onBeforeXR(mode: XRSessionMode, init: XRSessionInit)` | Called right before a XRSession is requested and can be used to modify the XRSessionInit object
| `onEnterXR(args: NeedleXREventArgs)` | Callback when this component joins a xr session (or becomes active in a running XR session)
| `onUpdateXR(args: NeedleXREventArgs)` | Callback when a xr session updates (while it is still active in XR session)
| `onLeaveXR(args: NeedleXREventArgs)` | allback when this component exists a xr session (or when it becomes inactive in a running XR session) 
| `onControllerAdded(args: NeedleXRControllerEventArgs)` | Callback when a controller is connected/added while in a XR session    OR when the component joins a running XR session that has already connected controllers   OR when the component becomes active during a running XR session that has already connected controllers
| `onControllerRemoved(args: NeedleXRControllerEventArgs)` | callback when a controller is removed while in a XR session   OR when the component becomes inactive during a running XR session

#### Additional XR events

| Method name | Description |
| -- | --
| `window.addEventListener("needle-xrsession-start")` | CustomEvent that is invoked when a XRSession starts. `details` contains the `NeedleXRSession`
| `window.addEventListener("needle-xrsession-end")` | CustomEvent that is invoked when a XRSession starts. `details` contains the `NeedleXRSession`
| `onXRSessionStart(args: { session:NeedleXRSession } )` | global event hook. To unsubscribe use `offXRSessionStart`


### Coroutines

Coroutines can be declared using the [JavaScript Generator Syntax](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator).  
To start a coroutine, call ``this.startCoroutine(this.myRoutineName());``  

**Example**
```ts twoslash
import { Behaviour, FrameEvent } from "@needle-tools/engine";

export class Rotate extends Behaviour {

    start() {
        // the second argument is optional and allows you to specifiy 
        // when it should be called in the current frame loop
        // coroutine events are called after regular component events of the same name
        // for example: Update coroutine events are called after component.update() functions
        this.startCoroutine(this.rotate(), FrameEvent.Update);
    }

    // this method is called every frame until the component is disabled
    *rotate() {
        // keep looping forever
        while (true) {
            yield;
        }
    }
}
```

To stop a coroutine, either exit the routine by returning from it, or cache the return value of ``startCoroutine`` and call ``this.stopCoroutine(<...>)``. All Coroutines are stopped at ``onDisable`` / when disabling a component.




## Special Lifecycle hooks

Needle Engine also exposes a few lifecycle hooks that you can use to hook into the update loop without having to write a full component.  
Those hooks can be inserted at any point in your web application (for example in toplevel scope or in a svelte component)  
| Method name | Description |
| -- | --
| `onInitialized(cb, options)` | Called when a new context is initialized (before the first frame)
| `onClear(cb, options)` | Register a callback before the engine context is cleared
| `onDestroy(cb, options)` |  Register a callback in the engine before the context is destroyed
| `onStart(cb, options)` | Called directly after components `start` at the beginning of a frame
| `onUpdate(cb, options)` | Called directly after components `update`
| `onBeforeRender(cb, options)` | called before calling render
| `onAfterRender(cb, options)` | called before calling render

For example ([See example on stackblitz](https://stackblitz.com/edit/needle-engine-lifecycle-hooks?file=src%2Fmain.ts))
```ts twoslash
// this can be put into e.g. main.ts or a svelte component (similar to onMount)
import { onStart, onUpdate, onBeforeRender, onAfterRender } from "@needle-tools/engine"

onStart(ctx => console.log("Hello Scene", ctx.scene));

onUpdate(ctx => {
    // do something... e.g. access the frame # or deltatime via ctx.time
    console.log("UPDATE", ctx.time.frame);
});

onBeforeRender(ctx => {
    // this event is only called once because of the { once: true } argument
    console.log("ON BEFORE RENDER", ctx.time.frame);
}, { once: true } );

// Every event hook returns a method to unsubscribe from the event
const unsubscribe = onAfterRender(ctx => {
    console.log("ON AFTER RENDER", ctx.time.frame);
});
// Unsubscribe from the event at any time
setTimeout(()=> unsubscribe(), 1000);
```

## Finding, adding and removing components

To access other components, use the static methods on ``GameObject`` or ``this.gameObject`` methods. For example, to access a `Renderer` component in the parent use ``GameObject.getComponentInParent(this.gameObject, Renderer)`` or ``this.gameObject.getComponentInParent(Renderer)``.  

**Example:**
```ts twoslash
import { Behaviour, GameObject, Renderer } from "@needle-tools/engine";

export class MyComponent extends Behaviour {

    start() {
        const renderer = GameObject.getComponentInParent(this.gameObject, Renderer);
        console.log(renderer);
    }
}
```




### Some of the available methods:


| Method |  |
| -- | -- 
| `GameObject.instantiate(Object3D, InstantiateOptions)` | creates a new instance of this object including new instances of all its components 
| `GameObject.destroy(Object3D \| Component)` | destroy a component or Object3D (and its components) 
| `GameObject.addNewComponent(Object3D, Type)` | adds (and creates) a new component for a type to the provided object. Note that ``awake`` and ``onEnable`` is already called when the component is returned 
| `GameObject.addComponent(Object3D, Component)` |  moves a component instance to the provided object. It is useful if you already have an instance e.g. when you create a component with e.g. `new MyComponent()` and then attach it to a object
| `GameObject.removeComponent(Component)` | removes a component from a gameObject
| `GameObject.getComponent(Object3D, Type)` | returns the first component matching a type on the provided object.
| `GameObject.getComponents(Object3D, Type)` | returns all components matching a type on the provided object.
| `GameObject.getComponentInChildren` | same as ``getComponent`` but also searches in child objects.
| `GameObject.getComponentsInChildren` | same as ``getComponents`` but also searches in child objects.
| `GameObject.getComponentInParent` | same as ``getComponent`` but also searches in parent objects.
| `GameObject.getComponentsInParent` | same as ``getComponents`` but also searches in parent objects.
| `GameObject.findObjectOfType` | searches the whole scene for a type.
| `GameObject.findObjectsOfType` | searches the whole scene for all matching types.


## Three.js and the HTML DOM

The context refers to the runtime inside a [web component](https://developer.mozilla.org/en-US/docs/Web/Web_Components).  
The three.js scene lives inside a custom HTML component called ``<needle-engine>`` (see the *index.html* in your project). You can access the `<needle-engine>` web component using ``this.context.domElement``.   

This architecture allows for potentially having multiple needle WebGL scenes on the same webpage, that can either run on their own or communicate between each other as parts of your webpage.  

### Access the scene
To access the current scene from a component you use `this.scene` which is equivalent to `this.context.scene`, this gives you the root three.js scene object.

To traverse the hierarchy from a component you can either iterate over the children of an object   
with a for loop:  
```ts twoslash
for(let i = 0; i < this.gameObject.children; i++) 
    console.log(this.gameObject.children[i]);
```
or you can iterate using the `foreach` equivalent:
```ts twoslash
for(const child of this.gameObject.children) {
    console.log(child);
}
```
You can also use three.js specific methods to quickly iterate all objects recursively using the [`traverse`](https://threejs.org/docs/#api/en/core/Object3D.traverse) method:  
```ts twoslash
import { Object3D } from "three";
this.gameObject.traverse((obj: Object3D) => console.log(obj));
```
or to just traverse visible objects use [`traverseVisible`](https://threejs.org/docs/#api/en/core/Object3D.traverseVisible) instead.

Another option that is quite useful when you just want to iterate objects being renderable you can query all renderer components and iterate over them like so:   
```ts twoslash
import { Renderer } from "@needle-tools/engine";
for(const renderer of this.gameObject.getComponentsInChildren(Renderer))
    console.log(renderer);
```
For more information about getting components see the next section.

### Time
Use `this.context.time` to get access to time data:  
- `this.context.time.time` is the time since the application started running
- `this.context.time.deltaTime` is the time that has passed since the last frame
- `this.context.time.frameCount` is the number of frames that have passed since the application started
- `this.context.time.realtimeSinceStartup` is the unscaled time since the application has started running  

It is also possible to use `this.context.time.timeScale` to deliberately slow down time for e.g. slow motion effects.

### Input
Receive input data for the object the component is on:
```ts twoslash
import { Behaviour } from "@needle-tools/engine";
export class MyScript extends Behaviour
{
    onPointerDown() {
        console.log("POINTER DOWN on " + this.gameObject.name);
    }
}
```

You can also subscribe to global events in the ``InputEvents`` enum like so:
```ts twoslash
import { Behaviour, InputEvents, NEPointerEvent } from "@needle-tools/engine";

export class MyScript extends Behaviour
{
    onEnable() {
        this.context.input.addEventListener(InputEvents.PointerDown, this.inputPointerDown);
    }

    onDisable() {
        // it is recommended to also unsubscribe from events when your component becomes inactive
        this.context.input.removeEventListener(InputEvents.PointerDown, this.inputPointerDown);
    }

    // @nonSerialized
    inputPointerDown = (evt: NEPointerEvent) => { console.log("POINTER DOWN anywhere on the <needle-engine> element"); }
}
```

Or use ``this.context.input`` if you want to poll input state every frame:

```ts twoslash
import { Behaviour } from "@needle-tools/engine";
export class MyScript extends Behaviour
{
    update() {
        if(this.context.input.getPointerDown(0)){
            console.log("POINTER DOWN anywhere")
        }
    }
}
```

If you want to handle inputs yourself you can also subscribe to [all events the browser provides](https://developer.mozilla.org/en-US/docs/Web/Events) (there are a ton). For example to subscribe to the browsers click event you can write:
```ts twoslash
import { Behaviour } from "@needle-tools/engine";
export class MyScript extends Behaviour
{
    onEnable() {
        window.addEventListener("click", this.windowClick);
    }

    onDisable() {
        // unsubscribe again when the component is disabled
        window.removeEventListener("click", this.windowClick);
    }

    windowClick = () => { console.log("CLICK anywhere on the page, not just on <needle-engine>"); }
}
```
Note that in this case you have to handle all cases yourself. For example you may need to use different events if your user is visiting your website on desktop vs mobile vs a VR device. These cases are automatically handled by the Needle Engine input events (e.g. `PointerDown` is raised both for mouse down, touch down and in case of VR on controller button down). 

### Raycasting

Use ``this.context.physics.raycast()`` to perform a raycast and get a list of intersections. If you dont pass in any options the raycast is performed from the mouse position (or first touch position) in screenspace using the currently active `mainCamera`. You can also pass in a `RaycastOptions` object that has various settings like `maxDistance`, the camera to be used or the layers to be tested against.

Use ``this.context.physics.raycastFromRay(your_ray)`` to perform a raycast using a [three.js ray](https://threejs.org/docs/#api/en/math/Ray).

> **Note**: This type of raycast casts a ray against all visible objects in the scene. No physics engine is needed, which is different to the behaviour in Unity, where you always need colliders to hit objects. If you want to cast against physics colliders only, use `physics.engine.raycast` methods described below.

#### Performance considerations

When using default Needle compression settings, simplified versions of meshes are automatically created and used for raycasting as well. Still, some types of meshes are slow – for example, skinned meshes or meshes with blendshapes require expensive calculations to determine exact hits. Consider setting those objects to the `Ignore Raycast` layer in Unity to avoid raycasting against them.

#### Physics-based Raycasting

Another option is to use the physics raycast methods which will only return hits with colliders in the scene. 

```ts twoslash
const hit = this.context.physics.engine?.raycast();
```

Here is an editable [example for physics raycast](https://stackblitz.com/edit/needle-engine-physics-raycast-example?file=src%2Fmain.ts,package.json,.gitignore)

### Networking
Networking methods can be accessed via ``this.context.connection``. Please refer to the [networking docs](./networking.md) for further information.


## Accessing Needle Engine and components from anywhere
It is possible to access all the functionality described above using regular JavaScript code that is not inside components and lives somewhere else. All the components and functionality of the needle runtime is accessible via the global ``Needle`` namespace (you can write ``console.log(Needle)`` to get an overview)

You can find components using ``Needle.findObjectOfType(Needle.AudioSource)`` for example. It is recommended to cache those references, as searching the whole scene repeatedly is expensive. See the list for [finding adding and removing components](#finding-adding-and-removing-components) above.  

For getting callbacks for the initial scene load see the following example:  
```js
<needle-engine loadstart="loadingStarted" progress="loadingProgress" loadfinished="loadingFinished"></needle-engine>

<script type="text/javascript">
function loadingStarted() { console.log("START") }
function loadingProgress() { console.log("LOADING...") }
function loadingFinished() { console.log("FINISHED!") }
</script>
```  

You can also subscribe to the globale `NeedleEngine` (sometimes also referred to as *ContextRegistry*) to receive a callback when a Needle Engine context has been created or to access all available contexts:
```ts twoslash
class YourComponentType extends Behaviour {}
//---cut---
import { NeedleEngine, GameObject, Behaviour } from "@needle-tools/engine";

NeedleEngine.addContextCreatedCallback((args) => {
  const context = args.context;
  const scene = context.scene;
  const myInstance = GameObject.getComponentInChildren(scene, YourComponentType);
});
```

Another option is using the `onInitialized(ctx => {})` [lifecycle hook](#special-lifecycle-hooks)

You can also access all available contexts via `NeedleEngine.Registered` which returns the internal array. (Note that this array should not be modified but can be used to iterate all active contexts to modify settings, e.g. set all contexts to `context.isPaused = true`) 

Below you find a list of available events on the static `NeedleEngine` type.   
You can subscribe to those events via `NeedleEngine.registerCallback(ContextEvent.ContextCreated, (args) => {})`

| ContextEvent options | |
|---|---|
| `ContextEvent.ContextRegistered` | Called when the context is registered to the registry. |
| `ContextEvent.ContextCreationStart` | Called before the first glb is loaded and can be used to initialize the physics engine. Can return a promise |
| `ContextEvent.ContextCreated` | Called when the context has been created before the first frame |
| `ContextEvent.ContextDestroyed` | Called when the context has been destroyed |
| `ContextEvent.MissingCamera` | Called when the context could not find a camera, currently only called during creation |
| `ContextEvent.ContextClearing` | Called when the context is being cleared: all objects in the scene are being destroyed and internal state is reset |
| `ContextEvent.ContextCleared` | Called after the context has been cleared |
 

## Gizmos

The static `Gizmos` class can be used to draw lines, shapes and text which is mostly useful for debugging.  
All gizmos function have multiple options for e.g. colors or for how long they should be displayed in the scene. Internally they are cached and re-used. 

| Gizmos | |
| -- | -- |
| `Gizmos.DrawLabel` | Draws a label with a background optionally. It can be attached to an object. Returns a Label handle which can be used to update the text. |
| `Gizmos.DrawRay` | Takes an origin and direction in worldspace to draw an infinite ray line |
| `Gizmos.DrawDirection` | Takes a origin and direction to draw a direction in worldspace |
| `Gizmos.DrawLine` | Takes two vec3 worldspace points to draw a line |
| `Gizmos.DrawWireSphere` | Draws a wireframe sphere in worldspace |
| `Gizmos.DrawSphere` | Draws a solid sphere in worldspace |
| `Gizmos.DrawWireBox` | Draws a wireframe box in worldspace |
| `Gizmos.DrawWireBox3` | Draws a wireframe box3 |
| `Gizmos.DrawArrow` | Draws an arrow taking two points in worldspace |


## Serialization / Components in glTF files
To embed components and recreate components with their correct types in glTF, we also need to save non-primitive types (everything that is not a ``Number``, ``Boolean`` or ``String``). You can do so is adding a ``@serializable(<type>)`` decorator above your field or property. 

**Example:**
@[code ts twoslash](@code/component-object-reference.ts)

To serialize from and to custom formats, it is possible to extend from the ``TypeSerializer`` class and create an instance. Use ``super()`` in the constructor to register supported types.  

> **Note**: In addition to matching fields, matching properties will also be exported when they match to fields in the typescript file. 

## Loading Scenes
In Unity referenced Prefabs, SceneAssets and AssetReferences (Unity's Addressable System) will automatically be exported as glTF files (please refer to the [Export Prefabs](export.md) documentation).  

These exported gltf files will be serialized as plain string URIs. To simplify loading these from TypeScript components, we added the concept of ``AssetReference`` types. They can be loaded at runtime and thus allow to defer loading parts of your app or loading external content.

**Example:**
@[code ts twoslash](@code/component-prefab.ts)

AssetReferences are cached by URI, so if you reference the same exported glTF/Prefab in multiple components/scripts it will only be loaded once and then re-used.  

# Next Steps

---
lang: en-US
title: Bike Configurator 🚲
sidebar: false
editLink: false
---

### Live

<sample src="https://bike.needle.tools" />

[Visit website](https://bike.needle.tools)

---
lang: en-US
title: Castle Builder 🏰
sidebar: false
editLink: false
---

### Live

<sample src="https://castle.needle.tools" />

[Visit website](https://castle.needle.tools)

### How To Play

Build your own castle! Drag 3D models from the various palettes onto the stage, and create your very own world.
Works on Desktop, Mobile, VR, AR, all right in your browser. Interactions are currently optimized for VR; placement on screens is a bit harder but possible. Have fun, no matter which device you're on!

Invite your friends! Click Create Room to be put into a live, multi-user space – just copy the URL, send it to a friend and they join you automatically.
There's currently a max limit for "users online at the same time" - if you don't get into a room, please try later.

### Info

This page was authored in Unity and exported to three.js using tools and technologies by 🌵 needle.

There are a lot of open technologies involved: 3D models are in glTF format, the render engine is three.js, VR and AR are using WebXR. The networking server runs on Glitch, and audio is sent over WebRTC using PeerJS.

---
lang: en-US
title: Mercedes-Benz Showcase
editLink: false
---

## About

Hello, my name is Kryštof and i did a research project about Needle. At [our company](https://www.ishowroom.cz/home/), we wanted to determine how Needle can help us in our workflow. We have one local client which focuses on reselling luxury cars. We already delivered a mobile app and VR experience using Unity. We have around 30 unique cars ready in the engine. We plan to expand the client's website with visually pleasing digital clones with more configuration options. Needle could achieve a perfect 1:1 conversion between unity and web visuals. It would be a massive benefit to our workflow. So that's what sparked our research.


<sample src="https://engine.needle.tools/demos/mercedes-benz-demo/" />


## Context

I'm not very well experienced with javascript, typescript or three.js, so my point of view is as a semi-experienced Unity developer trying out the simplest way how to create a web experience. For those who would suggest Unity WebGL, that sadly doesn't work and isn't flexible on mobile browsers. Needle is 💚


## Lighting

Our lighting model is based on reflection probes in unity. We do not need any directional or point lights, only ambient lighting. 


We're using this skybox:

 ![Skybox](/showcase-mercedes/1_skybox.png)

Which looks like this on the paint job:

![Paintjob](/showcase-mercedes/2_paintjob_simple.jpg)

Then to add a slight detail, i've added 2 directional lights with an insignificant intensity (0.04) to create specular highlights. So before it looked like this:

![Specular off](/showcase-mercedes/3_SpecularHighlights_off.jpg)

But with the added directional lights it added a better dynamic. The effect could be deepened with higher intensity:

![Specular on](/showcase-mercedes/4_SpecularHighlights_on.jpg)



## Background

The scene now looks like this:

![No background](/showcase-mercedes/5_NoBackground.jpg)

The black background isn't very pretty. So to differentiate between visual and lighting skyboxes i've added an inverse sphere which wraps the whole map.

![With background](/showcase-mercedes/6_MapBackground.png)

Regarding the gradient goes from a slight gray to a white color..

This effect could be easily made with just a proper UV mapping and a single pixel high texture which would define the gradient.

I've made an unlit shader in the shader graph:

![Evironemnt shader](/showcase-mercedes/7_EnvShaderGraph.jpg)

I've noticed a color banding issue, so i've tried to implement dithering. Frankly, it didn't help the artefacts but i bet there's a simple solution to that issue. So the upper part of the shader does sample the gradient based on the Y axis in object space. And the lower part tries to negate the color banding.

By using shaders it's simpler to use and iterate the gradiant. By using Needle's Shadergraph markdown asset, it's even simpler! 🌵

![Gradiant](/showcase-mercedes/8_Gradiant.png)


## Car fake movement

The scene right now is static since nothing moves. We can negate that by adding a fake feeling of motion. Let's start by adding motion to the wheels.

With a simple component called Rotator, we define an axis and speed along it.

![Rotator](/showcase-mercedes/9_Rotator.png)
```ts twoslash
import { Behaviour, serializable } from "@needle-tools/engine";

export enum RotationAxis {
    X, Y, Z
}

export class Rotator extends Behaviour {
    //@type RotationAxis
    @serializable()
    axis : RotationAxis = RotationAxis.X;

    @serializable()
    speed : number = 1;

    update() {
        const angle = this.speed * this.context.time.deltaTime;
        switch(this.axis) {
            case RotationAxis.X:
                this.gameObject.rotateX(angle);
                break;
            case RotationAxis.Y:
                this.gameObject.rotateY(angle);
                break;
            case RotationAxis.Z:
                this.gameObject.rotateZ(angle);
                break;
        }
    }
}
```


The user now sees a car driving in deep nothingness, the color doesn't resemble anything and the experience is dull. We want to ground the model and that's done by adding a grid and then shifting it so it seems the car is moving. This is what we want to achieve:

![Motion](/showcase-mercedes/10_WheelsAndGrid.png)

The shader for the grid was comprised of two parts. A simple tiled texture of the grid that's being multipled by a circular gradient to make the edges fade off.

![Grid](/showcase-mercedes/11_GridShader.jpg)


## Extra elements

This tech demo takes it's goal to showcase the car's capabilities.

Let's start by highlighting the wheels.

![Wheel highlight](/showcase-mercedes/12_WheelWithText.png)

Adding this shader to a plane will result in a dashed circle which is rotating by a defined speed. Combined with world space UI with a normal Text component this can highlight some interesting capabilities or parameters of the given product.

![Wheel shader](/showcase-mercedes/13_WheelShader.jpg)

After showcasing the wheels we want to finish with a broad information about the product. In this case, that would be the car's full name and perhaps some available configurations.

![Rear UI](/showcase-mercedes/14_RearUI.jpg)



## Wrap up

By using the Unity's timeline we can control when the wheel dashes and text will be shown. This is complemented by the camera animation.


## Conclusion

Needle Engine seems to be a very good candidate for us! 

There are a few features which we miss. 

That would be for example proper support for the Lit Shader Graphs. But nothing stops us to create shaders the three.js way and create simmilar shaders in Unity for our content team to tweak the materials.

Using Needle was a blast! 🌵

---
lang: en-US
title: Monster Hands 💀
sidebar: false
editLink: false
---

### Live

<sample src="https://monster-hands.needle.tools/" />

[Visit website](https://monster-hands.needle.tools/)

---
lang: en-US
title: Tower Defense
sidebar: false
editLink: false
---

### Live

<sample src="https://v6p9d9t4.ssl.hwcdn.net/html/7746989/index.html" />

[Visit website](https://willitaugment.itch.io/tumbleweed-defender)

---
lang: en-US
title: Castle Builder 🏰
sidebar: false
editLink: false
---

### Live

<sample src="https://needle.tools" />

[Visit website](https://needle.tools)

---
lang: en-US
title: Monster Hands 💀
sidebar: false
editLink: false
---

### Live

<sample src="https://zenrepublic.space/?realm=3" />

[Visit website](https://zenrepublic.space/?realm=3)

# Support and Community

## Community Forum

Ask questions and get help in [Needle Forum](https://forum.needle.tools)  

::: info Get instant help with Needle AI
Ask any question about Needle Engine, web development or our editor integrations and get instant help from the Needle AI that has access to the latest code, documentation and our integrations.
:::


## Needle Discord

Chat with the community and share your projects in [Needle Discord](https://discord.needle.tools/?utm_source=needle_docs&utm_content=content)


## Local AI

Needle Engine documentation is also available via the llms.txt standard.    
Download the files below to use as context for your local AI:

- [llms.txt](https://cloud.needle.tools/llms.txt)
- [llms-full.txt](https://cloud.needle.tools/llms-full.txt)


## Get Inspired

Watch some of our [user interviews on youtube](https://www.youtube.com/playlist?list=PLJ4BaFFEGP1EOHCjYszc__d2yO7RkB-iw) to get inspired.

<video-embed src="https://www.youtube.com/watch?v=naPlw5aDJHs" />

<video-embed src="https://www.youtube.com/watch?v=1KKfct3Zpcw" />


<video-embed src="https://www.youtube.com/watch?v=gZuC40Alr88" />


<video-embed src="https://www.youtube.com/watch?v=F6_buCHZhWk" />



::: info You made a cool project?
Don't hestitate to [reach out](mailto:hi@needle.tools) – we always love seeing what you built 💚 
:::

# Technical Overview

## How it works

Needle Engine roughly consists of three parts:
- a number of **components and tools** that allow you to set up scenes for Needle Engine from e.g. the Unity Editor.  
- an **exporter** that turns scene and component data into glTF.
- a **web runtime** that loads and runs the produced glTF files and their extensions.

The web runtime uses three.js for rendering, adds a component system on top of the three scene graph and hooks up extension loaders for our custom glTF extensions.  

Effectively, this turns tools like Unity or Blender into spatial web development powerhouses – adding glTF assets to the typical HTML, CSS, JavaScript and bundling workflow.  


## glTF Assets

Models, textures, animations, lights, cameras and more are stored as [glTF 2.0 files](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html) in Needle Engine.  
Custom data is stored in [vendor extensions](#vendor-specific-gltf-extensions-needle_). These cover everything from interactive components to physics, sequencing and lightmaps.  

### Supported glTF extensions

A typical production glTF created by Needle Engine uses the following extensions:  
```
KHR_lights_punctual
KHR_materials_unlit
KHR_texture_transform
KHR_animation_pointer
NEEDLE_techniques_webgl
NEEDLE_gameobject_data
NEEDLE_components
NEEDLE_persistent_assets
NEEDLE_lightmaps
NEEDLE_lighting_settings
KHR_texture_basisu
KHR_draco_mesh_compression
```

Other supported extensions:
```
EXT_meshopt_compression
EXT_mesh_gpu_instancing (import and export)
```

Supported material extensions:  

```
KHR_materials_clearcoat
KHR_materials_ior
KHR_materials_specular
KHR_materials_transmission
KHR_materials_iridescence
KHR_materials_unlit
KHR_materials_volume
```

More extensions and custom extensions can be added using the export callbacks of UnityGLTF (not documented yet) and the [glTF import extensions](https://threejs.org/docs/#examples/en/loaders/GLTFLoader) of three.js.  

> **Note**: Materials using these extensions can be exported from Unity via UnityGLTF's `PBRGraph` material.  

> **Note**: Audio and variants are already supported in Needle Engine through `NEEDLE_components` and `NEEDLE_persistent_assets`, but there are some options for more alignment to existing proposals such as `KHR_audio` and `KHR_materials_variants`.

[Learn more about GLTF loading in three.js](https://threejs.org/docs/#examples/en/loaders/GLTFLoader)

### Compression

For production, we compress glTF assets with [`glTF-transform`](https://gltf-transform.donmccurdy.com/). Textures use either `etc1s`, `uastc`, `webp` or no compression, depending on texture type. Meshes use `draco` by default but can be configured to use `meshtopt` (per glTF file). Custom extensions are passed through in an opaque way.  

See the [deployment & compression](./deployment.md#optimization-and-compression-options) page for more information


## Vendor-specific glTF Extensions (NEEDLE_*)

Needle Engine stores custom data in glTF files through our vendor extensions. These extensions are designed to be flexible and allow relatively arbitrary data to put into them. Notably, no code is stored in these files. Interactive components is restored from the data at runtime. This has some similarities to how AssetBundles function in Unity – the receiving side of an asset needs to have matching code for components stored in the file.  

> We're currently not prodiving schemas for these extensions as they are still in development. The JSON snippets below demonstrates extension usage by example and includes notes on architectural choices and what we may change in future releases.  

> References between pieces of data are currently constructed through a mix of indices into other parts of the glTF file and JSON pointers. We may consolidate these approaches in a future release. We're also storing string-based GUIDs for cases where ordering is otherwise hard to resolve (e.g. two components referencing each other) that we may remove in the future.  

### NEEDLE_components

This extension contains per-node component data. The component names map to type names on both the JavaScript and C# side.  
Multiple components with the same name can be added to the same node.  

Data in `NEEDLE_components` can be animated via the currently not ratified [`KHR_animation_pointer`](https://github.com/ux3d/glTF/tree/extensions/KHR_animation_pointer/extensions/2.0/Khronos/KHR_animation_pointer) extension.  

```json
"NEEDLE_components": {
  "builtin_components": [
    {
      "name": "WebARSessionRoot",
      "guid": "1516450550",
      "arScale": 50,
      "invertForward": true,
      "enabled": true,
      "gameObject": {
        "node": 0
      }
    },
    {
      "name": "SyncedRoom",
      "guid": "1516450552",
      "roomName": "network-room",
      "urlParameterName": "room",
      "joinRandomRoom": true,
      "requireRoomParameter": false,
      "autoRejoin": true,
      "enabled": true,
      "gameObject": {
        "node": 0
      }
    },
    {
      "name": "PlayableDirector",
      "guid": "2243275882009986562_1668529989451832962",
      "state": 0,
      "extrapolationMode": 1,
      "playableAsset": "extensions/NEEDLE_persistent_assets/4",
      "playableGraph": {},
      "playOnAwake": true,
      "timeUpdateMode": 0,
      "time": 0,
      "initialTime": 0,
      "duration": 135.383333333332,
      "enabled": true,
      "gameObject": {
        "node": 0
      }
    }
  ]
}
```

> **Note**: Storing only the component type name means that type names currently need to be unique per project. We're planning to include package names in a future release to loosen this constraint to unique component type names per package instead of globally.  

> **Note**: Currently there's no versioning information in the extension (which npm packaage does a component belong to, which version of that package was it exported against). We're planning to include versioning information in a future release.  

> **Note**: Currently all components are in the `builtin_components` array. We might rename this to just `components` in a future release.  

### NEEDLE_gameobject_data

This extension contains additional per-node data related to state, layers, and tags. Layers are used for both rendering and physics, similar to how [three.js](https://threejs.org/docs/#api/en/core/Layers) and [Unity](https://docs.unity3d.com/Manual/Layers.html) treat them.  

```json
"NEEDLE_gameobject_data": {
  "layers": 0,
  "tag": "Untagged",
  "hideFlags": 0,
  "static": false,
  "activeSelf": true,
  "guid": "1516450549"
}
```

> **Note**: We may need to better explain why this is not another entry in [`NEEDLE_components`](#needle_components). 

### NEEDLE_lighting_settings

This is a root extension defining ambient lighting properties per glTF file.   

```json
"NEEDLE_lighting_settings": {
  "ambientMode": 0,
  "ambientLight": [
    0.212,
    0.227,
    0.259,
    1
  ],
  "ambientIntensity": 1,
  "defaultReflectionMode": 0
}
```

> **Note**: This extension might have to be defined per-scene instead of per-file.

### NEEDLE_lightmaps

This is a root extension defining a set of lightmaps for the glTF file.

```json
"NEEDLE_lightmaps": {
  "textures": [
    {
      "pointer": "textures/20",
      "type": 1,
      "index": 0
    }
  ]
}
```

> **Note**: At the moment this extension also contains environment texture references. We're planning to change that in a future release. 

| Texture Type | Value |
| -- | -- |
| Lightmap | 0 |
| Environment Map  | 1 |
| Reflection Map | 2 |

How lightmaps are applied is defined in the `MeshRenderer` component inside the [`NEEDLE_components`](#needle_components) extension per node:  

```json
"NEEDLE_components": {
  "builtin_components": [
    {
      "name": "MeshRenderer",
      ...
      "lightmapIndex": 0,
      "lightmapScaleOffset": {
        "x": 1.00579774,
        "y": 1.00579774,
        "z": -0.00392889744,
        "w": -0.00392889744
      },
      ...
    }
  ]
}
```

> **Note**: We may change that in a future release and move lightmap-related data to a `NEEDLE_lightmap` extension entry per node. 

### NEEDLE_persistent_assets

Components in `NEEDLE_components` can reference data via JSON Pointers. The data in `NEEDLE_persistent_assets` is often referenced multiple times by different components and is thus stored separately in a root extension. By design, they are always referenced by something else (or have references within each other), and thus do not store type information at all: they're simply pieces of JSON data and components referencing them currently need to know what they expect. 

Examples for assets/data stored in here are:  
- AnimatorControllers, their layers and states
- PlayableAssets (timelines), their tracks and embedded clips
- SignalAssets
- ...

Data in `persistent_assets` can reference other `persistent_assets` via JSON Pointer, but by design can't reference `NEEDLE_components`. This is similar to the separation beween "Scene data" and "AssetDatabase content" in Unity. 

```json
{
  "name": "LampionController",
  "guid": "9100000_ecab75bc7ab51a747a4c5c14236a43cd",
  "parameters": [],
  "layers": [
    {
      "name": "Base Layer",
      "stateMachine": {
        "name": "Base Layer",
        "defaultState": 0,
        "states": [
          {
            "name": "LampionFlying",
            "hash": 677739540,
            "motion": {
              "name": "LampionFlying",
              "isLooping": false,
              "guid": "7400000_c296c4d76e956b34f8b5833ba90653c1",
              "clips": [
                {
                  "node": "nodes/4",
                  "clip": "animations/0"
                },
                {
                  "node": "nodes/9",
                  "clip": "animations/6"
                },
                {
                  "node": "nodes/14",
                  "clip": "animations/12"
                }
              ]
            },
            "transitions": [
              {
                "isExit": false,
                "exitTime": 1,
                "hasFixedDuration": true,
                "offset": 0,
                "duration": 0,
                "hasExitTime": true,
                "destinationState": 0,
                "conditions": []
              }
            ]
          }
        ],
        "entryTransitions": []
      }
    }
  ]
},
{
  "name": "TrongCom Website",
  "guid": "11400000_93a8f856fe26af8498d94efe4835af36",
  "tracks": [
    {
      "name": "Markers",
      "type": "MarkerTrack",
      "muted": false,
      "outputs": [
        null
      ],
      "clips": [],
      "markers": [],
      "trackOffset": null
    },
    {
      "name": "Animation Track",
      "type": "AnimationTrack",
      "muted": false,
      "outputs": [
        "5017454109690854928_1668529989451832962"
      ],
      "clips": [
        {
          "start": 0,
          "end": 0.9833333333333333,
          "duration": 0.9833333333333333,
          "timeScale": 1,
          "asset": {
            "clip": "animations/78",
            "loop": false,
            "duration": 8,
            "position": {
              "x": 0,
              "y": 0,
              "z": 0
            },
            "rotation": {
              "x": 0,
              "y": 0,
              "z": 0,
              "w": 1
            },
            "removeStartOffset": true
          },
          "clipIn": 0,
          "easeInDuration": 0,
          "easeOutDuration": 0.41666666666666663,
          "preExtrapolationMode": 1,
          "postExtrapolationMode": 1
        },
        ... 
      ]
    }
  ]
}
```

> **Note**: We might include more type and versioning information in the future. 

### NEEDLE_techniques_webgl

This extension builds upon the archived [`KHR_techniques_webgl`](https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Archived/KHR_techniques_webgl) extension and extends it in a few crucial places. While the original extension was specified against WebGL 1.0, we're using it with WebGL 2.0 here and have added a number of  uniform types.  

```json
"KHR_techniques_webgl": {
  "programs": [
    {
      "vertexShader": 1,
      "fragmentShader": 0,
      "id": 0
    }
  ],
  "shaders": [
    {
      "name": "Pass-FRAGMENT",
      "type": 35632,
      "uri": "<embedded WebGL fragment shader code ...>",
      "id": 1
    },
    {
      "name": "Pass-VERTEX",
      "type": 35633,
      "uri": "<embedded WebGL vertex shader code ...>",
      "id": 0
    }
  ],
  "techniques": [
    {
      "program": 0,
      "attributes": {},
      "uniforms": {
        "_TimeParameters": {
          "name": "_TimeParameters",
          "type": 35666,
          "semantic": null,
          "count": 1,
          "node": 0
        },
        "hlslcc_mtx4x4unity_MatrixVP": {
          "name": "hlslcc_mtx4x4unity_MatrixVP",
          "type": 35666,
          "semantic": "_VIEWPROJECTION",
          "count": 4,
          "node": 0
        }
      },
      "defines": []
    }
  ]
}     
```

> **Note**: Currently, vertex and fragment shaders are always embedded as URI; we plan on moving that data into more reasonable bufferViews in the future.  

> **Note**: There's some redundant properties in here that we plan on cleaning up.  

## TypeScript and Data Mapping

> 🏗️ Under Construction

## Rendering with three.js

> 🏗️ Under Construction

## Why aren't you compiling to WebAssembly?

While Unity's compilation process from C# to IL to C++ (via IL2CPP) to WASM (via emscripten) is ingenious, it's also relatively slow. Building even a simple project to WASM takes many minutes, and that process is pretty much repeated on every code change. Some of it can be avoided through clever caching and ensuring that dev builds don't try to strip as much code, but it still stays slow.  
> We do have a prototype for some WASM translation, but it's far from complete and the iteration speed stays slow, so we are not actively investigating this path right now. 

When looking into modern web workflows, we found that code reload times during development are neglectible, usually in sub-second ranges. This of course trades some performance (interpretation of JavaScript on the fly instead of compiler optimization at build time) for flexibility, but browsers got really good at getting the most out of JavaScript.  

We believe that for iteration and tight testing workflows, it's beneficial to be able to test on device and on the target platform (the browser, in this case) as quickly and as often as possible - which is why we're skipping Unity's entire play mode, effectively always running in the browser. 

> **Note**: A really nice side effect is avoiding the entire slow "domain reload" step that usually costs 15-60 seconds each time you enter Play Mode. You're just "live" in the browser the moment you press Play.

---
next: getting-started/
---

# Testimonials

<p></p>

<testimonial
  name="Chris Mahoney"
  role="Founder, Our Computer Company. Ex-Unity, Ex-Lamina1"
  src="https://x.com/mahoneymatic/status/1562981022932684800?t=qNqojoZkk2CZrJa7dGzqng&s=19"
  img="https://pbs.twimg.com/profile_images/1872967967404642304/kOGAWAzd_x96.jpg"
>
Unbelievable Unity editor integration by an order of magnitude,  
  and as straightforward as the docs claim. Wow.
</testimonial>

<testimonial 
  name="Kevin Curry" 
  role="Founder, Intangible.ai. Ex-Unity"
  src="https://x.com/kmcurry/status/1574333302022062080"
  img="https://pbs.twimg.com/profile_images/1773236097642123264/nBGahkSO_x96.jpg"
>
needle.tools is a wonderful showcase of what @NeedleTools contributes to 3D via the web. I just love it.
</testimonial>

<testimonial
  name="Brit Gardner"
  role="Senior Software Engineer, Coherence. Ex-Blizzard"
  src="https://x.com/britg/status/1562443905580163072"
  img="https://pbs.twimg.com/profile_images/1360002897048727553/0gscFjvt_x96.jpg"
>
Played with this a bit this morning 🤯🤯 pretty magical
</testimonial>

<testimonial
  name="Marc Wakefield"
  role="Augmented Reality Expert"
  src="https://x.com/mrm_design/status/1567391880169545729"
  img="https://pbs.twimg.com/profile_images/1106863126958493697/tlgb3EyM_x96.jpg"
>
This is huge for WebXR and shared, immersive 3D experiences! Thank you so much for putting in the work on this @NeedleTools crew! Hoping @Apple
 sort out their WebXR situation sooner rather than later. The AR part worked flawlessly on my @SamsungMobile S21.
</testimonial>

<testimonial
  name="Rinesh Thomas"
  role="Game Developer, Assassin's Creed VR"
  src="https://x.com/rineshthomas/status/1566342798063947777?t=z6sG3Z7mol-NfIRfTTKqCQ&s=19"
  img="https://pbs.twimg.com/profile_images/1427818935/rinesh_small_x96.jpg"
>
    This is the best thing I have seen after cinemachine in unity. Unity should acquire this
</testimonial>

<testimonial
  name="Stella Cannefax"
  role="CTO Dora, Game Developer"
  src="https://x.com/0xstella/status/1574853012585172993"
  img="https://pbs.twimg.com/profile_images/1657086025661833237/Yyo56hVY_x96.jpg"
>
Thanks to @NeedleTools, seeing quite a bit of this solution for web-based real time 3d tools - export scenes from Unity, where you can leverage the extensive 3d editor ecosystem & content, and then render them in your own web-based engine
</testimonial>

<testimonial
  name="Pete Patterson" 
  role="Spatial Computing Specialist"
  src="https://x.com/VRSpatialist/status/1572300394285383680"
  img="https://pbs.twimg.com/profile_images/1560370896866533378/aEzq15Cm_x96.jpg"
>
Finally checking out @NeedleTools with Unity. Super easy to get something up and running in the cloud using their
 integrations
</testimonial>

<testimonial 
  name="Dilmer Valecillos" 
  role="Developer Advocate for Mixed Reality @Meta, Tutorial Creator"
  src="https://x.com/Dilmerv/status/1562209049856188420"
  img="https://pbs.twimg.com/profile_images/1876790676610023424/4KHnReTp_x96.jpg"
>
 This is amazing and if you are curious about #WebXR with Unity this will help us get there
</testimonial>

<testimonial 
  name="VRSpatialist" 
  src="https://discord.com/channels/717429793926283276/722046635525537842/1030201907513405530"
  img="https://cdn.discordapp.com/avatars/563549194137239552/b0e0746e19ad79db53cf4465ebb0791e.webp?size=96"
>
 I am a long time Unity dev and recently started playing with Needle Tools and I love it! It's a great on ramp for Unity devs who want to learn WebXR and three.js. The runtime engine is awesome and it was pretty easy to create my own custom component
</testimonial>

<testimonial
  name="Unity for Digital Twins"
  role="Powering the enterprise metaverse"
  src="https://x.com/DigitalTwin/status/1576934958681055233"
  img="https://pbs.twimg.com/profile_images/1659581582657863681/xl4kQKA__x96.jpg"
>
We just gotta say WOW 🤩
</testimonial>

<testimonial
  name="Matthew Pieri" 
  role="Game Developer"
  src="https://discord.com/channels/717429793926283276/1097572505738301571/1097572505738301571"
  img="https://pbs.twimg.com/profile_images/1109900618657878016/3cej9Jql_x96.jpg"
>
Spent the last 2.5 months building this game, never built a game/never used unity before, but absolutely loving the whole process with needle tools. So rapid!  Would love to make a career building AR experiences! 
</testimonial>

<testimonial 
  name="Yuzu" 
  role="Web Game Developer"
  src="https://discord.com/channels/717429793926283276/1264966222467043433/1265268833485066240"
  img="https://cdn.discordapp.com/avatars/559315446105636875/c85b503132ed62d741a37cf9174646d2.webp?size=96"
>
My workflow has been optimized 10X ever since i started using needle
</testimonial>

---
title: Testing on local devices
---

## Testing on local devices

When using our templates, Needle Engine runs a local development server for you. Simply press play, and a website will open in your default browser, ready for testing on your local device. For testing on other devices in the same network, you may have to install a self-signed certificate (see below).

When using a custom setup / framework, please refer to your framework's documentation on how to run a secure local development server.  

::: tip
Our local server uses the IP address in your local network (e.g. `https://192.168.0.123:3000`) instead of `localhost:3000`.  This allows you to quickly view and test your project from mobile devices, VR glasses, and other machines in the same network.   

We're using HTTPS instead of the older HTTP, because modern powerful web APIs like WebXR require a secure connection – the S stands for "secure".
:::

## Setting up a self-signed certificate for development

Different operating systems have different security requirements for local development. Typically, displaying a website will work even with a auto-generated untrusted certificate, but browsers may warn about the missing trust and some features are not available. Here's a summary:

::: tip
Installing trusted self-signed certificates on your development devices is recommended for a smooth development experience. Find the steps at the bottom of this page.
:::

**Default – with auto-generated untrusted certificate**  
_Displays a certificate warning upon opening the project in a browser._
_Uses the [vite-plugin-basic-ssl](https://github.com/vitejs/vite-plugin-basic-ssl) npm package._

We're using websocket connections to communicate between the browser and the local development server. Websockets require a secure connection (HTTPS) in order to work. Depending on the platform, you might need to set up a custom certificate for local development. Android and Windows don't need a custom certificate, but iOS and MacOS do.

| OS | Viewing in the browser<br/>(with a security warning) | Automatic reloads |
| --- | --- | --- |
| Windows | (✓) | ✓ |
| Linux | (✓) | ✓ |
| Android | (✓) | ✓ |
| macOS | (✓) | ❌ |
| iOS | (✓ Safari and Chrome, after page reload)<br/>❌ Mozilla XR Viewer | ❌ |
| Xcode Simulators | (✓ after page reload) | ❌ |

**With a self-signed, trusted root certificate**  
_No security warning is displayed. You need to install the generated certificate on your device(s)._  
_Uses the [vite-plugin-mkcert](https://github.com/liuweiGL/vite-plugin-mkcert) npm package._


| OS | Viewing in the browser | Automatic reloads |
| --- | --- | --- |
| Windows | ✓ | ✓ |
| Linux | ✓ | ✓ |
| Android | ✓ | ✓ |
| macOS | ✓ | ✓ |
| iOS | ✓ | ✓ |

### Generating a self-signed development certificate

- in Unity/Blender, click on "Open Workspace" to open VS Code  

- check that you're using `vite-plugin-mkcert` instead of `vite-plugin-basic-ssl` (the latter doesn't generate a trusted root certificate, which we need) in your `vite.config.ts` file:
  - open `package.json`
  - remove the line with `"@vitejs/plugin-basic-ssl"` from `devDependencies`
  - open a Terminal in VS Code and run `npm install vite-plugin-mkcert --save-dev` which will add the latest version
  - open `vite.config.ts` and replace `import basicSsl from '@vitejs/plugin-basic-ssl'` with `import mkcert from'vite-plugin-mkcert'`
  - in `plugins: [`, replace `basicSsl(),` with `mkcert(),`
  - package.json looks like this now: 
  ![](/testing/switch-to-mkcert.webp)
- run `npm run start` once from VS Code's terminal
  - on Windows, this will open a new window and guide you through the creation and installation of the certificate
  - on MacOS, the terminal prompts for your password and then generates and installs the certificate.
  - if you're getting an error `Error: Port 3000 is already in use`, please close the server that may still be running from Unity.
- the generated certificate will automatically be installed on the machine you generated it on.
- you can stop the terminal process again.
- from now on, pressing Play in Unity/Blender will use the generated certificate for the local server, and no "security warning" will be shown anymore, since your browser now trusts the local connection.

## Installing the certificate on your development devices

On your development devices, you need to _install_ the generated certificate and allow the OS to _trust_ it. This is different for each OS. For each of them, you'll need the rootCA.pem file that was generated, and send it to the device you want to authenticate.

**On Windows:** find the certificate in `Users/<your-user>/.vite-plugin-mkcert/rootCA.pem`  
**On MacOS:** find the certificate in `Users/<your-user>/.vite-plugin-mkcert/rootCA.pem`  

Send the device to yourself (e.g. via E-Mail, AirDrop, iCloud, USB, Slack, ...) so that you can access it on your development devices.

### Installing the certificate on Android

1. Open the file. You'll be prompted to install the certificate.

### Installing the certificate on iOS / iPadOS / VisionOS
1. Open the file.
2. You'll be prompted to _add_ the profile to your device. Confirm.
3. Go to Settings
4. There will be a new entry "Profile". Select it and allow the profile to be _active_ for this device.
5. On iOS / iPadOS, you also need to allow "Root Certificate Trust". For this, search for `Trust` or go to `Settings > General > About > Info > Certificate Trust Settings` and enable full trust for the root certificate.

::: tip
The certificate is automatically installed on the machine you generated it on. For other machines in the local network, follow the steps below to also establish a trusted connection.
:::

### Installing the certificate on another MacOS machine
1. Open the file. Keychain Access will open and allow you to install the certificate. 
2. You may have to set "Trust" to "Always allow".

### Installing the certificate on another Windows machine
1. Open `certmgr` ("Manage user certificates") by typing <kbd>Windows key</kbd> + `certmgr`.
2. In the left sidebar, select "Trusted Root Certification Authorities".
3. Right-click on "Certificates" and select "All Tasks > Import".
4. Select the `rootCA.pem` file (you may have to change the file type to "all") and follow the instructions.

---
title: Using Needle Engine directly from HTML
---


This page has been moved into the [getting started](./getting-started/) guide

---
next: features-overview
---

# Our Vision 🔮

## The Future of the 3D Web

We believe the use of 3D on the web will expand considerably in the next years. While today native apps are the norm, more and more content is made available as a web app or [PWA](https://web.dev/progressive-web-apps/).  New VR and AR devices will [extend into the web](https://immersive-web.github.io/webxr-samples/), creating an interesting problem: responsive suddenly not only means  "small screen" or "large screen", you're also dealing with spaces, 3D, spatial placement and potentially glasses and controllers!  

Add to that a push towards more interactivity and collaboration, and you have an interesting mix of challenges.  

At Needle, we believe ideating and creating in this space should be easy. We've set out to speed things up – creating our own runtime to reach these goals. That's why we're baking the ability to deploy to AR and VR right into our core components, and continually test that new ideas work across platforms. 

## Why another platform for 3D on the web? Aren't there enough options already?

There's numerous options, that's true! We found that current systems<sup>1</sup> can be roughly sorted into two categories: some have great asset handling, tools, and artist-friendly workflows but output some sort of binary blob, and others are more code-focussed, developer-friendly and allow for great integration into modern web workflows<sup>2</sup>.  

We want to bridge these worlds and combine the best of both worlds: artist-friendly workflows and modern web technologies. Combined with modern formats and a snappy workflow, we believe this will allow many more creators to bring their content to the web. We also saw an opportunity to get AR, VR and collaboration right from the start.  
  
<sup>1</sup>: _Examples include Unity, PlayCanvas, three.js, react-three-fiber, Babylon, A-Frame, Godot, and many more._
<sup>2</sup>: _There's more nuance to this than fits into an introductory paragraph! All engines and frameworks have their strengths and weaknesses, and are constantly evolving._
  
## Creating a Workflow, not an Editor
  
We think the next wave of 3D apps on the web will come with better _workflows_: everyone should be able to put together a 3D scene, an art gallery, present a product or 3D scan on the web or make simple games. Reaching this goal will require more than just supporting one particular system and exporting to the web from there.
  
Our goal is to allow people to bring data to the web from _their_ creative tools: be it Unity, Blender, Photoshop, or something else. We're aware that this is a big goal – but instead of doing everything at once, we want to iterate and get closer to it together.  
  
## Open Standards instead of Proprietary Containers

At the core of Needle Engine stands the [glTF](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html) format and its ability to be extended with custom extensions. The goal is: a single `.glb` file can contain your entire application's data. 
  
It's worth noting that it's not a goal to ship actual code inside glTF; shipping and running code is the job of modern web runtimes and bundling. We certainly can imagine that abstract representations of logic (e.g. graphs, state machines, and so on) can be standardized to a certain degree and allow for interoperable worlds, but we're not there yet.  
  
[Read more about our use of glTF and extensions](./technical-overview.md)

# Goals and Non-Goals

## Goals
- Iteration should be rapid and deployment should be fast. 
- Working on 3D web projects should be the as easy as working 2D web projects.   
- Developers and artists should be able to collaborate directly.   
- Responsive web extends beyond screens – AR and VR should be built in, not afterthoughts.   
- We want to contribute back to open-source projects. 
- Open discussion regarding 3D and web standards. 
- Ability to bring and take your data in open formats. 
- Ability to choose what web framework you use, not lock-in to particular frameworks and vendors. 
- Common usecases work without or with limited coding experience.  

## Non-Goals
- It's not a goal to have 100% coverage of all combinations of Editor versions, feature sets, render pipelines.  
- It's not a goal to provide a full no-code environment.  
- It's not a goal to match the feature set, capabilities, or runtime performance of other engines.  
  
# Relation to other engines and frameworks  

## Needle Engine and Unity WebGL

From working with Unity for many years we've found that while the engine and editor progress at a great pace, WebGL output has somewhat lacked behind. Integration of Unity players into web-based systems is rather hard, "talking" to the surrounding website requires a number of workarounds, and most of all, iteration times are very slow due to the way that Unity packs all code into WebAssembly via IL2CPP. These technologies are awesome, and result in great runtime performance and a lot of flexibility. But they're so much slower and walled off compared to modern web development workflows that we decided to take matters into our own hands.  

## Needle Engine and three.js

Needle Engine builds on three.js. All rendering goes through it, glTF files are loaded via three's extension interfaces, and our component system revolves around three's Object3D and scene graph. We're committed to upstreaming some of our changes and improvements, creating pull requests and reporting issues along the way.

---
title: VR & AR (WebXR)
---

## Supported Devices

Needle Engine supports the full [WebXR specification](https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API), including AR and VR. WebXR is an official web standard that brings immersive experiences to the web, with all the benefits of the web: no installation, no app store, no SDKs required.

All devices with a browser can run apps made with Needle. If the browser supports WebXR, your apps will automatically work in XR as well, using our built-in components. This includes desktop browsers, mobile browsers, many browsers on AR/VR headsets, but also other emerging technologies like Looking Glass displays, smart glasses, and more.

:::tip App-free iOS AR support via USDZ/QuickLook
While iOS devices don't yet have official WebXR support, Needle supports creating AR experiences on iOS using [Everywhere Actions](everywhere-actions.md). See the [iOS section](#augmented-reality-and-webxr-on-ios) for more details. You can create rich, interactive experiences that work seamlessly in AR on iOS devices, even with the limitations that Apple has in place.   

When you enter AR mode on iOS, Needle will automatically convert your scene to an USDZ file, which is then displayed in AR using Apple's QuickLook. Objects, materials, audio, animation and Everywhere Actions will be preserved. 
:::

The following table lists some of the devices that we verified to work with Needle Engine.
When a new device comes out that supports WebXR, it will work with your apps out of the box. This is one of the big advantages of building with the browser as a platform – compatibility is not limited to a specific set of devices or SDKs.


| Headset Device | Browser | Notes |
| -- | -- | -- |
| Apple Vision Pro | ✔️ Safari | hand tracking, support for transient pointer |
| Meta Quest 3 | ✔️ Meta Browser | hand tracking, support for sessiongranted<sup>1</sup>, passthrough, depth sensing, mesh tracking |
| Meta Quest 3S | ✔️ Meta Browser | hand tracking, support for sessiongranted<sup>1</sup>, passthrough, depth sensing, mesh tracking |
| Meta Quest 2 | ✔️ Meta Browser | hand tracking, support for sessiongranted<sup>1</sup>, passthrough (black and white) |
| Meta Quest 1 | ✔️ Meta Browser | hand tracking, support for sessiongranted<sup>1</sup> |
| Meta Quest Pro | ✔️ Meta Browser | hand tracking, support for sessiongranted<sup>1</sup>, passthrough |
| Pico Neo 4 | ✔️ Pico Browser | passthrough, hand tracking<sup>2</sup> |
| Pico Neo 3 | ✔️ Pico Browser | no hand tracking, inverted controller thumbsticks |
| Oculus Rift 1/2 | ✔️ Chrome |  |
| Valve Index | ✔️ Chrome |  |
| HTC Vive | ✔️ Chrome |  |
| Hololens 2 | ✔️ Edge | hand tracking, support for AR and VR (in VR mode, background is rendered as well) |

| Mobile Device | Browser | Notes |
| -- | -- | -- |
| Android 10+ | ✔️ Chrome | |
| Android 10+ | ✔️ Firefox | |
| iOS 15+ | (✔️)<sup>3</sup> Safari<br/>(✔️)<sup>3</sup> Chrome | No full code support, but Needle [Everywhere Actions](everywhere-actions.md) are supported for creating dynamic, interactive USDZ files. |
| iOS 15+ | ✔️ WebXR Viewer | browser is somewhat dated by now |
| Hololens 2 | ✔️ Edge | |
| Hololens 1 | ❌ | no WebXR support |
| Magic Leap 2 | ✔️ | |
| Magic Leap 1 | ✔️ | deprecated device |

| Other Devices | Browser | Notes |
| -- | -- | -- |
| Looking Glass Holographic Display | ✔️ Chrome | requires Looking Glass bridge and some custom code, [see our sample](https://engine.needle.tools/samples/looking-glass/) |
| Logitech MX Ink | ✔️ Meta Browser | officially supported, see [docs](https://logitech.github.io/mxink/WebXR/WebXrIntegration.html#using-needle-tools) |

<sup>1</sup>: Requires enabling a browser flag: `chrome://flags/#webxr-navigation-permission`   
<sup>2</sup>: Requires enabling a toggle in the Developer settings    
<sup>3</sup>: Uses [Everywhere Actions](everywhere-actions.md) or [other approaches](#augmented-reality-and-webxr-on-ios)

## VR, AR and QuickLook Examples

Visit our [Needle Engine Samples](https://engine.needle.tools/samples/?overlay=samples&tag=xr) to try many interactive examples right now. Or, try it live on your device by clicking the <kbd>QR Code</kbd> (for phones) or <kbd>Open on Quest</kbd> (for Meta Quest headsets) buttons below.

<sample src="https://engine.needle.tools/samples/collaborative-sandbox/"/>

## Adding VR and AR capabilities to a scene

AR, VR and networking capabilites in Needle Engine are designed to be modular. You can choose to not support any of them, or add only specific features. 

### Basic capabilities

1. **Enable AR and VR**  
  Add a `WebXR` component.  
  *Optional:* you can set a custom avatar by referencing an [Avatar Prefab](#avatars).   
  By default, a basic `DefaultAvatar` is assigned.
  
2. **Enable Teleportation**  
  Add a `TeleportTarget` component to object hierarchies that can be teleported on.  
  To exclude specific objects, set their layer to `IgnoreRaycasting`.  

### Multiplayer

1. **Enable Networking**  
  Add a `SyncedRoom` component.

2. **Enable Desktop Viewer Sync**  
  Add a `SyncedCamera` component.
  
3. **Enable Voice Chat**  
  Add a `VoIP` component.

:::tip Scene structure
These components can be anywhere inside your hierarchy. They can also all be on the same GameObject, which is a common pattern.
:::

 > **[Castle Builder](https://castle.needle.tools/)** uses all of the above for a cross-platform multiplayer sandbox experience.   
 > — #madebyneedle 💚  
   
### Special AR Components

1. **Define the AR Session root and scale**  
  Add a `WebARSessionRoot` component to your root object. For AR experiences, often you want to scale the scene to fit the real world.  
2. Define the **user scale** to shrink (< 1) or enlarge (> 1) the user in relation to the scene when entering AR.

### Controlling object display for XR

1. **Define whether an object is visible in Browser, AR, VR, First Person, Third Person**  
  Add a `XR Flag` component to the object you want to control.

2. **Change options on the dropdown** as needed.   
    Common usecases are
    - hiding floors when entering AR
    - hiding Avatar parts in First or Third Person views. For example, in first-person view a person shouldn't be able to see their own head model.

### Travelling between VR worlds

Needle Engine supports the [`sessiongranted`](https://github.com/immersive-web/navigation) state. This allows users to seamlessly traverse between WebXR applications without leaving an immersive session – they stay in VR or AR.  

Currently, this is only supported on Oculus Quest 1, 2 and 3 in the Oculus Browser. On other platforms, users will be kicked out of their current immersive session and have to enter VR again on the new page.  
Requires enabling a browser flag: `chrome://flags/#webxr-navigation-permission`  

- **Click on objects to open links**  
  Add the `OpenURL` component that makes it very easy to build connected worlds.  

## Scripting  
Read more about scripting for XR at the [scripting XR documentation](./scripting.md#xr-event-methods)

## Avatars

While we don't currently provide an out-of-the-box integration external avatar systems, you can create application-specific avatars or custom systems.  

- **Create a custom Avatar**  
  - Create an empty GameObject as avatar root
  - Add an object named `Head` and add a `XRFlag` that's set to Third Person
  - Add objects named `HandLeft` and `HandRight`
  - Add your graphics below these objects.

### Experimental Avatar Components

There's a number of experimental components to build more expressive Avatars. At this point we recommended duplicating them to make your own variants, since they might be changed or removed at a later point.  

![20220817-230858-87dG-Unity_PLjQ](https://user-images.githubusercontent.com/2693840/185243523-57c4b2a9-0ec7-4f88-b53b-585e879d504d.gif)  
*Example Avatar Rig with basic neck model and limb constraints*

- **Random Player Colors**  
  As an example for avatar customization, you can add a `PlayerColor` component to your renderers.  
  This randomized color is synchronized between players.  

- **Eye Rotation**  
  `AvatarEyeLook_Rotation` rotates GameObjects (eyes) to follow other avatars and a random target. This component is synchronized between players.  
  
- **Eye Blinking**  
  `AvatarBlink_Simple` randomly hides GameObjects (eyes) every few seconds, emulating a blink.  
  
  ![image](https://user-images.githubusercontent.com/2693840/185233753-e6de49f0-31c3-4851-9919-551309303ebd.png)  
  *Example Avatar Prefab hierarchy*
  
- **Offset Constraint**  
  `OffsetConstraint` allows to shift an object in relation to another one in Avatar space. This allows, for example, to have a Body follow the Head but keep rotation levelled. It also allows to construct simple neck models.  
  
- **Limb Constraint**  
  `BasicIKConstraint` is a very minimalistic constraint that takes two transforms and a hint. This is useful to construct simple arm or leg chains. As rotation is currently not properly implemented, arms and legs may need to be rotationally symmetric to "look right". It's called "Basic" for a reason!  

## HTML Content Overlays in AR  
    
If you want to display different html content whether the client is using a regular browser or using AR or VR, you can just use a set of html classes.  
This is controlled via HTML element classes. For example, to make content appear on desktop and in AR add a ``<div class="desktop ar"> ... </div>`` inside the `<needle-engine>` tag:  

```html
<needle-engine>
    <div class="desktop ar" style="pointer-events:none;">
        <div class="positioning-container">
          <p>your content for AR and desktop goes here</p>
          <p class="only-in-ar">This will only be visible in AR</p>
        <div>
    </div>
</needle-engine>
```

Content Overlays are implemented using the optional `dom-overlay` feature which is usually supported on screen-based AR devices (phones, tablets).  

Use the `.ar-session-active` class to show/hide specific content while in AR. The [`:xr-overlay` pseudo class](https://www.w3.org/TR/webxr-dom-overlays-1/#css-pseudo-class) shouldn't be used at this point because using it breaks Mozilla's WebXR Viewer. 

```css
.only-in-ar {
  display: none;
}

.ar-session-active .only-in-ar {
  display:initial;
}
```

It's worth noting that the overlay element [will be always displayed fullscreen while in XR](https://www.w3.org/TR/webxr-dom-overlays-1/#ua-style-sheet-defaults), independent of styling that has been applied. If you want to align items differently, you should make a container _inside_ the `class="ar"` element.  

<sample src="https://engine.needle.tools/samples-uploads/ar-overlay/"/>


## Augmented Reality and WebXR on iOS

Augmented Reality experiences on iOS are somewhat limited, due to Apple currently not supporting WebXR on iOS devices.  

Needle Engine's [Everywhere Actions](everywhere-actions.md) are designed to fill that gap, bringing automatic interactive capabilities to iOS devices for scenes composed of specific components. They support a subset of the functionality that's available in WebXR, for example spatial audio, image tracking, animations, and more. See [the docs](everywhere-actions.md) for more information. 

:::tip Limited custom code support in QuickLook
Apple has strong limitations in place what kind of content can be used in QuickLook. Thus, custom script components can not automatically be converted for use in AR on iOS. You can add support for some sorts of custom code using our Everywhere Actions API.
:::

### Musical Instrument – WebXR and QuickLook support

Here's an example for a musical instrument that uses Everywhere Actions and thus works in browsers and in AR on iOS devices. 
It uses spatial audio, animation, and tap interactions.  
<sample src="https://engine.needle.tools/samples-uploads/musical-instrument" />

### Everywhere Actions and other options for iOS AR

There's also other options for guiding iOS users to even more capable interactive AR experiences:

3. **Exporting content on-the-fly as USDZ files.**  
   These files can be displayed on iOS devices in AR. When exported from scenes with Everywhere Actions the interactivity is the same, more than sufficient for product configurators, narrative experiences and similar.
   An example is [Castle Builder](https://castle.needle.tools) where creations (not the live session) can be viewed in AR.  

 > **[Encryption in Space](https://accurate-tree-observation.glitch.me/)** uses this approach. Players can collaboratively place text into the scene on their screens and then view the results in AR on iOS. On Android, they can also interact right in WebXR.   
 > — #madewithneedle by Katja Rempel 💚  

1. **Guiding users towards WebXR-compatible browsers on iOS.**
   Depending on your target audience, you can guide users on iOS towards for example Mozilla's [WebXR Viewer](https://apps.apple.com/de/app/webxr-viewer/id1295998056) to experience AR on iOS.  
   
2. **Using camera access and custom algorithms on iOS devices.**  
   One can request camera image access and run custom algorithms to determine device pose.  
   While we currently don't provide built-in components for this, here's a few references to libraries and frameworks that we want to try in the future:  
   - [AR.js](https://github.com/AR-js-org/AR.js) (open source)
     - [Experimental AR.js integration](https://github.com/FireDragonGameStudio/NeedleAndARjs) by FireDragonGameStudio
   - [Mind AR](https://github.com/hiukim/mind-ar-js) (open source)
   - [8th Wall](https://www.8thwall.com/) (commercial)

## Image Tracking

Needle Engine supports **WebXR Image Tracking** ([Live Demo](https://engine.needle.tools/samples/image-tracking?utm_source=docs&utm_content=xr)) on Android and **QuickLook Image Tracking** on iOS.

You can find additional documentation in the [Everywhere Actions](everywhere-actions.md#image-tracking) section.

:::warning WebXR Image Tracking is still in a "draft" phase and not generally available
So far, browser vendors haven't been able to agree on the final image tracking API for WebXR. As long as the specification is in "draft" phase ([Marker Tracking Explainer](https://github.com/immersive-web/marker-tracking/blob/main/explainer.md)),
you and your app's users need to follow these steps to enable WebXR ImageTracking on Android devices:
1. Visit ``chrome://flags`` on your Android Chrome browser
2. Find and enable the `WebXR Incubations` option
:::


Without that spec, one can still request camera image access and run custom algorithms to determine device pose. The downside is that users will have to accept additional permissions like camera access, and the tracking will not be as accurate as with the native capabilities of the device.

Here are some libraries to add image tracking based on camera access and local computer vision algorithms:  
   - [Experimental AR.js integration with Needle Engine](https://github.com/FireDragonGameStudio/NeedleAndARjs) by FireDragonGameStudio
   - [AR.js](https://github.com/AR-js-org/AR.js) (open source)
   - [Mind AR](https://github.com/hiukim/mind-ar-js) (open source)



## References

[WebXR Device API](https://www.w3.org/TR/webxr/)  
[caniuse: WebXR](https://caniuse.com/webxr)  
[Apple's Preliminary USD Behaviours](https://developer.apple.com/augmented-reality/quick-look/)

---
title: needle.config.json
---

The `needle.config.json` is used to provide configuration for the Needle Editor integrations and for the Needle Engine build pipeline plugins.

| | |
| --- | --- | 
| **Paths** | |
| `buildDirectory` | This is where the built project files are being copied to |
| `assetsDirectory` | This is where the Editor integration assets will be copied to or created at (e.g. the `.glb` files exported from Unity or Blender) |
| `scriptsDirectory` | This is the directory the Editor integration is watching for code changes to re-generate components |
| `codegenDirectory` | This is where the Editor integration is outputting generated files to. |
| `baseUrl` | Required for e.g. next.js or SvelteKit integration. When baseUrl is set, relative paths for codegen and inside files are using baseUrl, not assetsDirectory. This is useful in cases where the assetDirectory does not match the server url.<br/>For example, the path on disk could be `"assetsDirectory": "public/assets"`, but the framework serves files from `"baseUrl": "assets"`. |
| **Tools** | |
| `build : { copy: ["myFileOrDirectory"] }` | Array of string paths for copying additional files or folders to the `buildDirectory`. These can either be absolute or relative. |


#### Basic Example 
```json
{
  "buildDirectory": "dist",
  "assetsDirectory": "assets",
  "scriptsDirectory": "src/scripts",
  "codegenDirectory": "src/generated"
}
```

#### Copy Example
```json
{
  "buildDirectory": "dist",
  "assetsDirectory": "assets",
  "scriptsDirectory": "src/scripts",
  "codegenDirectory": "src/generated",
  "build": {
    "copy": [
      "cards"
    ]
  }
}
```

#### Example with different baseUrl (e.g. SvelteKit, Next.js)
Files are exported to `static/assets` but the framework serves them from `/assets`. In this case, the `baseUrl` needs to be set to `assets` so that relative paths in files are correct.

```json
{
  "baseUrl": "assets",
  "buildDirectory": "dist",
  "assetsDirectory": "static/assets",
  "scriptsDirectory": "src/scripts",
  "codegenDirectory": "src/generated"
}
```

#### Related Links
- [Project Structure](../project-structure.md)

---
title: <needle-engine> Configuration
---

The `<needle-engine>` web-component comes with a nice collection of built-in attributes that can be used to modify the look and feel of the loaded scene without the need to add or edit the three.js scene directly.   
The table below shows a list of the most important ones:

| Attribute | Description |
| --- | --- |
| **Loading** | |
| `src` | Path to one or multiple glTF or glb files.<br/>Supported types are `string`, `string[]` or a stringified array (`,` separated) |
| `dracoDecoderPath` | URL to the draco decoder |
| `dracoDecoderType` | draco decoder type. Options are `wasm` or `js`. See [three.js documentation](https://threejs.org/docs/#examples/en/loaders/DRACOLoader.setDecoderConfig) |
| `ktx2DecoderPath` | URL to the KTX2 decoder |
| **Rendering** | |
| `background-color` | optional, hex color to be used as a background color. Examples: `rgb(255, 200, 100)`, `#dddd00` | 
| `background-image` | optional, URL to a skybox image (background image) or a preset string: `studio`, `blurred-skybox`, `quicklook`, `quicklook-ar` | 
| `background-blurriness` | optional, bluriness value between 0 (no blur) and 1 (max blur) for the `background-image`. Example: `background-blurriness="0.5"` | 
| `environment-image` | optional, URL to a environment image (environment light) or a preset string: `studio`, `blurred-skybox`, `quicklook`, `quicklook-ar` |
| `contactshadows` | optional, render contact shadows |
| `tone-mapping` | optional, supported values are `none`, `linear`, `neutral`, `agx` |
| `tone-mapping-exposure` | optional number e.g. increase exposure with `tone-mapping-exposure="1.5"`, requires `tone-mapping` to be set |
| **Interaction** | |
| `autoplay` | add or set to `true` to auto play animations e.g. `<needle-engine autoplay` | 
| `camera-controls` | add or set to `true` to automatically add OrbitControls if no camera controls are found in the scene |
| `auto-rotate` | add to enable auto-rotate (only used with `camera-controls`) |
| **Events** | |
| `loadstart` | Name of the function to call when loading starts. Note that the arguments are `(ctx:Context, evt:Event)`. You can call `evt.preventDefault()` to hide the default loading overlay | 
| `progress` | Name of the function to call when loading updates. `onProgress(ctx:Context, evt: {detail: {context:Context, name:string, index:number, count:number, totalProgress01:number}) { ... }`   |
| `loadfinished` | Name of the function to call when loading finishes | 
| **Loading Display** | *Available options to change how the Needle Engine loading display looks. Use `?debugloadingrendering` for easier editing* |
| `loading-style` | Options are `light` or `dark` |
| `loading-background-color` | **PRO** — Change the loading background color (e.g. `=#dd5500`) |
| `loading-text-color` | **PRO** — Change the loading text color |
| `loading-logo-src` | **PRO** — Change the loading logo image |
| `primary-color` | **PRO** — Change the primary loading color |
| `secondary-color` | **PRO** — Change the secondary loading color |
| `hide-loading-overlay` | **PRO** — Do not show the loading overlay, added in Needle Engine > 3.17.1
| **Internal** | |
| `hash` | Used internally, is appended to the files being loaded to force an update (e.g. when the browser has cached a glb file). Should not be edited manually. |


# Examples

```html
<!-- Setting the path to a custom glb to be loaded -->
<needle-engine src="path/to/your.glb"></needle-engine>
```

```html
<!-- Overriding where the draco decoder is located -->
<needle-engine src="path/to/your.glb" dracoDecoderPath="path/to/draco/folder"></needle-engine>
```

Setting environment images, playing animation and automatic camera controls. [See it live on stackblitz](https://stackblitz.com/edit/needle-engine-cycle-src?file=index.html)
```html
<needle-engine
      camera-controls
      auto-rotate
      autoplay
      skybox-image="https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/industrial_sunset_puresky_1k.hdr"
      environment-image="https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/industrial_sunset_puresky_1k.hdr"
      src="https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/DamagedHelmet/glTF-Embedded/DamagedHelmet.gltf"
      >
      </needle-engine>
```

Receiving an event when the needle-engine context has finished loading:
```html
<needle-engine loadfinished="onLoadFinished"> </needle-engine>
<script>
    function onLoadFinished() {
        console.log("Needle Engine has finished loading");
    }
</script>
```

### Custom Loading Style (PRO)

You can easily modify how Needle Engine looks by setting the appropriate attributes on the `<needle-engine>` web component. Please see the table above for details.

![custom loading](/imgs/custom-loading-style.webp)  
[See code on github](https://github.com/needle-engine/vite-template/blob/loading-style/custom/index.html)

---
title: "@serializable and other decorators"
---

The following table contains available Typescript decorators that Needle Engine provides.  
 
You can think of them as Attributes on steroids (if you are familiar with C#) - they can be added to classes, fields or methods in Typescript to provide additional functionality. 

|  |  | 
| --- | --- |
| **Field & Property Decorators** | |
| `@serializable()` | Add to exposed / serialized fields. Is used when loading glTF files that have been exported with components from Unity or Blender. |
| `@syncField()` | Add to a field to network the value when it changes. You can pass in a method to be called when the field changes |
| `@validate()` | Add to receive callbacks in the component event method `onValidate` whenever the value changes. This behaves similar to Unity's onValidate. |
| **Method Decorators** | |
| `@prefix(<type>)` (experimental) | Can be used to easily inject custom code into other components. Optionally return `false` to prevent the original method from being executed. See the [example below](#prefix) |
| **Class Decorators** | |
| `@registerType` | No argument. Can be added to a custom component class to be registered to the Needle Engine types and to enable hot reloading support. |


## Examples


### Serializable

```ts twoslash
import { Behaviour, serializable, EventList } from "@needle-tools/engine";
import { Object3D } from "three";

export class SomeComponentType extends Behaviour {}

export class ButtonObject extends Behaviour {
    // you can omit the type if it's a primitive 
    // e.g. Number, String or Bool
    @serializable()
    myNumber: number = 42;

    // otherwise add the concrete type that you want to serialize to
    @serializable(EventList)
    onClick?: EventList;

    @serializable(SomeComponentType)
    myComponent?: SomeComponentType;

    // Note that for arrays you still add the concrete type (not the array)
    @serializable(Object3D)
    myObjects?: Object3D[];
}
```


### SyncField

The `@syncField` decorator can be used to automatically network properties of your components for all users (visitors of your website) connected to the same networking room. It can optionally take a callback function that will be invoked whenever the value changes.     

- To notify the system that a reference value (like an object or an array) has changed you need to re-assign the field. E.g. like this: `myField = myField`   
- The callback function can *not* be an arrow function (e.g. `MyScript.prototype.onNumberChanged` works for `onNumberChanged() { ... }` but it does not for `myNumberChanged = () => { ... }`)

```ts twoslash
import { Behaviour, serializable, syncField } from "@needle-tools/engine";

export class MyScript extends Behaviour {

    @syncField(MyScript.prototype.onNumberChanged)
    @serializable()
    myNumber: number = 42;

    private onNumberChanged(newValue: number, oldValue: number){
        console.log("Number changed from ", oldValue, "to", newValue)
    }
}
```


### Validate
```ts twoslash
import { Behaviour, serializable, validate } from "@needle-tools/engine";

export class MyScript extends Behaviour {

    @validate()
    @serializable()
    myNumber?: number;

    start() { setInterval(() => this.myNumber = Math.random(), 1000) }

    onValidate(fieldName: string) {
        console.log("Validate", fieldName, this.myNumber);
    }
}
```


### Prefix
[Live example](https://stackblitz.com/edit/needle-engine-prefix-example?file=src%2Fmain.ts)
```ts twoslash
import { Camera, prefix } from "@needle-tools/engine";
class YourClass {
    @prefix(Camera) // < this is type that has the method you want to change
    awake() { // < this is the method name you want to change

        // this is now called before the Camera.awake method runs
        // NOTE: `this` does now refer to the Camera instance and NOT `YourClass` anymore. This allows you to access internal state of the component as well
        console.log("Hello camera:", this)
        // optionally return false if you want to prevent the default behaviour
    }
}
```

<br/>

<div class="centered" style="display: flex;
    align-items: center;
    gap: 20px;
    font-size: 2em;
    font-weight: 100;">
    <img src="/logo.png" style="max-height:70px;" title="Needle Logo" alt="Needle Logo"/> +
    <img src="/imgs/logo-webcomponents.png" style="max-height:70px;" title="Web Components Logo" alt="Web Components Logo"/> +
    <img src="/imgs/threejs-logo.webp" style="max-height:70px;" title="three.js Logo" alt="three.js Logo"/>
</div>

# Needle Engine as Web Component

Needle Engine provides an easy-to-use web component that can be used to display rich, interactive 3D scenes directly in HTML with just a few lines of code. It's the same web component that powers our integrations. 

Once you outgrow the configuration options of the web component, you can extend it with custom scripts and components, and full programmatic scene graph access.

:::tip Use the integrations!
For complex 3D scenes and fast iteration, we recommend using Needle Engine with one of our integrations. They provide a powerful creation workflow, including a live preview, hot reloading, and an advanced build pipeline with 3D optimizations.
:::

### Quick Start
:::: code-group
::: code-group-item index.html
@[code html](@code/basic-webcomponent.html)
:::
::: code-group-item Result
<iframe src="/docs/code-samples/basic-webcomponent.html" style="
    width: 100%; 
    aspect-ratio: 16/9; 
    outline: none; 
    border: none;
    "
    allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture; xr-spatial-tracking"
    allowfullscreen
    ></iframe>
:::
::::
[Open this example on Stackblitz](https://stackblitz.com/edit/needle-engine-prebundled?file=index.html)



## Install from npm

You can work directly with Needle Engine without using any Integration. Needle Engine uses [three.js](https://threejs.org/) as scene graph and rendering library, so all functionality from three.js is available in Needle as well.  

You can install Needle Engine from [`npm`](https://www.npmjs.com/package/@needle-tools/engine) by running:   
<br/>
`npm i @needle-tools/engine`  

## Install needle-engine from a CDN

While our default template uses [vite](https://vitejs.dev), Needle Engine can also be used directly with vanilla Javascript – without using any bundler.  

You can add a complete, prebundled version of Needle Engine to your website with just a line of code.
This includes our core components, physics, particles, networking, XR, and more. Use this if you're not sure!

```js
<script type="module" 
    src="https://cdn.jsdelivr.net/npm/@needle-tools/engine/dist/needle-engine.min.js">
</script>
```

If you know your project doesn't require physics features, you can also use a smaller version of Needle Engine, without the physics engine. This will reduce the total downloaded size. 
```js
<script type="module"
    src="https://cdn.jsdelivr.net/npm/@needle-tools/engine/dist/needle-engine.light.min.js">
</script>
```


Many examples can be found on [StackBlitz](https://stackblitz.com/@marwie/collections/needle-engine).  

## Rapid Prototyping on StackBlitz

For quick experiments, we provide a convenient link to create a new project ready to start: [engine.needle.tools/new](https://engine.needle.tools/new)

## Local Development with VS Code

If you want to work with Needle Engine without any integration, then you'll likely want to run a local server for your website. Here's how you can do that with Visual Studio Code:

1. Open the folder with your HTML file in Visual Studio Code.
2. Install the [LiveServer extension](https://marketplace.visualstudio.com/items?itemName=ritwickdey.LiveServer).  
3. Activate live-server (there's a button "Go Live" in the footer of VSCode) 
4. Open ``http://localhost:5500/index.html`` in your web browser, if it doesn't open automatically.


## three.js and Needle Engine

Since Needle Engine uses [three.js](https://threejs.org/) as scene graph and rendering library, all functionality from three.js is available in Needle as well and can be used from component scripts. We're using a fork of three.js that includes additional features and improvements, especially in relation to WebXR, Animation, and USDZ export.


::: tip
Make sure to update the ``<needle-engine src="myScene.glb">`` path to an existing glb file 
or [download this sample glb](https://github.com/needle-tools/needle-engine-samples/raw/main/vanilla/myScene.glb) and put it in the same folder as the index.html, name it ``myScene.glb`` or update the src path.
:::

@[code](@code/basic-html.html) 


[View on github](https://github.com/needle-tools/needle-engine-samples/tree/main/vanilla)

# Editor Sync

Needle allows for a very fast, iterative workflow between Unity and the browser. Usually, exports take less than a few seconds. However, once scenes become more complex, for some types of changes (adjusting material properties, nudging objects around), we provide an even faster way to see your changes – **Editor Sync**.

## How to use Editor Sync

You can enable Editor Sync by adding the `EditorSync` component to your scene. This component will connect your Unity Editor with your browser project and automatically sync applicable changes between the two.

:::tip
**Editor Sync** is currently an experimental feature. Please let us know about your experience with it! We're eager to hear your feedback.
:::

## Video Tutorial

This tutorial shows the Editor Sync workflow in action:

<video-embed src="https://www.youtube.com/watch?v=gZX_sqrne8U" limit_height />

---
title: Needle Engine for Unity
editLink: true
---
<br/>
<div class="centered" style="display: flex;
    align-items: center;
    gap: 20px;
    font-size: 2em;
    font-weight: 100;">
    <img src="/logo.png" style="max-height:70px;" title="Needle Logo" alt="Needle Logo"/> +
  <img src="/imgs/unity-logo.webp" style="max-height:70px;" />
</div>

# Needle Engine for Unity

Needle Engine for Unity allows you to create highly interactive, flexible and lightweight web applications right inside Unity. Use the powerful tools of the Unity editor to visually setup your 3D scenes, animate and design. Needle Engine for Unity takes care of exporting your scene to glTF and integrates easily with any web frontend framework.


## Install the Unity Package


<NoDownloadYet>
  <br/>
  <needle-button 
    event_goal="download_unity" 
    event_position="getting_started" 
    large 
    href="https://engine.needle.tools/downloads/unity?utm_source=needle_docs&utm_content=getting_started"
    same_tab
    next_url="/docs/unity/"
    >
    <strong>Download Needle Engine for Unity</strong>
  </needle-button> 
</NoDownloadYet>

<!-- [Mirror](https://package-installer.glitch.me/v1/installer/needle/com.needle.engine-exporter?registry=https://packages.needle.tools&scope=com.needle&scope=org.khronos)    -->

1. **Drop the downloaded .unitypackage file** into a Unity project and confirm that you want to import it.

2. **Wait a moment** for the installation and import to finish. A window may open stating that "A new scoped registry is now available in the Package Manager.". This is our Needle Package registry. You can safely close that window.  
3. **Explore Samples**.  
  Select the menu option `Needle Engine > Explore Samples` to view, open and modify all available [sample scenes](https://engine.needle.tools/samples).  

## Quickstart Video Tutorial

<video-embed src="https://www.youtube.com/watch?v=3dB-d1Jo_Mk" limit_height />

## Start from a Sample

There are 100+ samples that cover a wide range of topics, use cases, and industries.  
For a quick overview, take a look at our [Samples page](https://engine.needle.tools/samples/). 

All of these samples are available directly in Unity:
1. Go to `Needle Engine > Explore Samples` to browse for samples
2. Click "Install Samples" to install the samples package right inside your editor (or [download the samples unitypackage](http://engine.needle.tools/downloads/unity/samples) to manually install the package)
3. Choose any sample and click on `Open Scene`. 

:::tip The Samples are read-only – that makes them easy to update.
Our samples scenes are part of a UPM package in Unity. This means that you can't edit the assets and scripts in them directly – they are read-only. To edit an asset from the samples package, copy it into your project's `Assets` folder. To edit a script from the samples package, copy it into your web project's `src` folder.
::: 

## Start from a template

We provide a number of Scene Templates for quickly starting new projects.  
These allow you to go from idea to prototype in a few clicks.  

1. Click on `File > New Scene`

2. Select one of the templates with (needle) in their name and click `Create`.   
   We recommend the [Collaborative Sandbox](https://engine.needle.tools/samples/collaborative-sandbox) template which is a great way to get started with interactivity, multiplayer, and adding assets.  
3. Click Play to install and startup your new web project.

![20220822-140539-wqvW-Unity_oC0z-needle](https://user-images.githubusercontent.com/2693840/185917275-a147cd90-d515-4086-950d-78358185b1ef.png)


## Start from scratch

If you don't want to start from a scene template, you can follow these steps.  
Effectively, we're going to recreate the "Minimal (Needle)" template that's shipping with the package.  

1. **Create a new empty scene**  

2. **Set up your scene for exporting**   
  Add an empty GameObject, name it "Exporter" and add the `Needle Engine` component to it (formerly named `Export Info`).  
  In this component you create and quickly access your exported runtime project.  
  It also warns you if any of our packages and modules are outdated or not locally installed in your web project.  

    ::: tip Project Name and Scene Name
    By default, the project name matches the name of your scene. If you want to change that, you can pick or enter a ``Directory Name`` where you want to create your new web project. The path is relative to your Unity project.  
    :::
 
3. **Choose a web project template**
  Now, select a web project template for your project. The default template is based on [Vite](https://vitejs.dev/), a fast web app bundler.  
  <br/>
    ![Unity ExportInfo local templates](/imgs/unity-project-local-template.jpg)


4. Click Play to install and start your new web project


:::tip Define your own templates
If you find yourself creating many similar projects, you can create your own local or remote templates using the Project View context menu under `Create/Needle Engine/Project Template`. Templates can either be local on disk (a folder being copied) or remote repositories (a git repository being cloned).
:::

## Project Folders and Files


| Folder | |
| --- | --- |
| **Unity** | |
| `Assets` | This is where project specific/exclusive assets live. |
| `Packages` | This is where packages installed for this project live. A package can contain any asset type. The main difference is that it can be added to multiple Unity projects. It therefor is a great method to share code or assets. To learn more about packages see [the Unity documentation about packages](https://docs.unity3d.com/Manual/PackagesList.html).
| **Needle Engine Unity Package** | |
| ``Core/Runtime/Components`` | Contains all Needle Engine built-in components. Learn more about them in the [Components Reference](./../component-reference.md). |

-----

When creating a new web project in Unity, you can choose to create it from a local template (by default we ship a vite based web template). 

You can also reference remote templates by entering a repository URL in the ExportInfo project path (this can be saved with your scene for example). When creating a new web project the repository will be either cloned or downloaded (depending on if you have git installed) and searched for a `needle.config.json` file. If none can be found in the cloned repository the root directory will be used. Examples of remote template projects can be found on [github.com/needle-engine](https://github.com/needle-engine)

![Unity ExportInfo local templates](/imgs/unity-project-remote-template.jpg)

### Temporary Projects

If you're planning to only add custom files via NpmDefs and not change the project config (e.g. for a quick fullscreen test), you can prefix the project path with `Library`. The project will be generated in the Unity Project Library and does not need to be added to source control (the Library folder should be excluded from source control). We call these projects _temporary projects_. They're great for quickly testing out ideas!


## Typescript in Unity

**NPM Definition** are [npm packages](https://docs.npmjs.com/about-packages-and-modules) tightly integrated into the Unity Editor which makes it easily possible to share scripts with multiple web- or even Unity projects.    

C# component stubs for typescript files will also be automatically generated for scripts inside npmdef packages.

#### Creating and installing a npmdef
To create a *NPM Definition* right click in the Unity Project browser and select ``Create/NPM Definition``.   
You can **install a *NPM Definition* package** to your runtime project by e.g. selecting your ``Export Info`` component and adding it to the ``dependencies`` list (internally this will just add the underlying npm package to your package.json).

![image](https://user-images.githubusercontent.com/5083203/170374130-d0e32516-a1d4-4903-97c2-7ec9fa0b17d4.png)

Don't forget to install the newly added package by e.g. clicking Install on the ExportInfo component and also restart the server if it is already running

To edit the code inside a *NPM Definition* package just double click the asset *NPM Definition* asset in your project browser and it will open the vscode workspace that comes with each npmdef.


# Next Steps

- [Concept: Web Projects](../project-structure.md)
- [Concept: Exporting Assets](../export.md)
- [Concept: Deployment (Share you website)](../deployment.md)
- [Components: Learn about Everywhere Actions](../everywhere-actions.md)
- [Beginner Scripting: Typescript essentials](../getting-started/typescript-essentials.md)
- [Beginner Scripting: How to write custom components](../scripting.md)

# Code Of Conduct

## Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, political party, or sexual identity and orientation. Note, however, that religion, political party, or other ideological affiliation provide no exemptions for the behavior we outline as unacceptable in this Code of Conduct.

## Our Standards

Examples of behavior that contributes to creating a positive environment include:

- Using welcoming and inclusive language
- Being respectful of differing viewpoints and experiences
- Gracefully accepting constructive criticism
- Focusing on what is best for the community
- Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

- The use of sexualized language or imagery and unwelcome sexual attention or advances
- Trolling, insulting/derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or electronic address, without explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

## Scope

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team by DM at [Needle Discord](https://discord.needle.tools). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html

[homepage]: https://www.contributor-covenant.org

# HOW TO

## Development setup

### Initial Setup

- copy .env.template to .env
- set the correct values in .env, namely GITHUB_ACCESS_TOKEN
- get a Github access token from https://github.com/settings/tokens?type=beta with access to https://github.com/needle-tools/needle-engine-support
  - only needs permissions for discussions

### Local Development

- open the VS Code workspace
- run `npm install`
- run `npm run start`

### Adding new pages

- in documentation/.vuepress/config.js add a new entry to the sidebar and navbar

## Inject Sample Code

You can automatically include code written in the Needle Engine Samples repository in the documentation. The code for parsing the sample code can be found in [`documentation/.vuepress/plugins/include-samples-code/index.js`](documentation/.vuepress/plugins/include-samples-code/index.js)    

All typescript, javascript and c# code is parsed in the [Needle Engine Samples Repository](https://github.com/needle-tools/needle-engine-samples).   

Follow these steps to insert code blocks:

1) In the *samples repository* mark the beginning and end of a sample code block like this. Give each tag a sensible name, for example:    
      ```ts
      // START MARKER play_animation_on_trigger`
      <code>
      // END MARKER play_animation_on_trigger
      ```
2) Push to the `docs/code-marker` branch
3) In the *documentation repository* (this repo) include code blocks by adding a HTML comment in the markdown files. For example:  
      ```html
      <!-- SAMPLE subscribe_to_events -->      
      ```
      Note: If your code block doesnt show up check the console of the documentation local server. If you just updated the samples repository it may take a few minutes to be available.   
      
      You can also insert additional markdown inside of the HTML comment. It will then only be rendered if the sample code can be found.
      For example:  
      ```html
      <!-- SAMPLE disable environment light 
      ## Disable environment light
      -->
      ```
      This is looking for a sample marker with "disable environment light" and if it finds it, it will render the markdown in the subsequent rows and then the code

<br/>
<p align="center">
    <img width="120" src="https://engine.needle.tools/branding/needle-logo.png" alt="logo" />
</p>
<h2 align="center">
     Needle Engine
</h2>

<div align="center">
  
[![Version](https://img.shields.io/npm/v/@needle-tools/engine?style=flat&colorA=999&colorB=999)](https://www.npmjs.com/package/@needle-tools/engine)
[![Downloads](https://img.shields.io/npm/dt/@needle-tools/engine.svg?style=flat&colorA=999&colorB=999)](https://www.npmjs.com/package/@needle-tools/engine)
[![Discord Shield](https://img.shields.io/discord/717429793926283276?style=flat&colorA=999&colorB=999&label=discord&logo=discord&logoColor=ffffff)](https://discord.needle.tools)
[![Forum Shield](https://img.shields.io/badge/forum-forum.needle.tools-blue?style=flat&colorA=999&colorB=999)](https://forum.needle.tools)
</div>

<div align="center">
  
[Getting Started](https://docs.needle.tools/getting-started/) — [Samples](https://engine.needle.tools/samples) — [Documentation](https://engine.needle.tools/docs)

</div>

---
    
**Needle Engine** is a web-based runtime for 3D apps. It runs on your machine for development, and can be deployed anywhere. It is flexible, extensible and has built-in support for networking and XR! 

**Needle Exporter for Unity** bridges the Unity Editor and the web runtime. It helps you to export your assets, animations, lightmaps and so on to the web. It is built around the glTF standard for 3D assets. 

**Together**, they enable incredible iteration speeds and help you to bring your content to the web.  
Some have called it the "Missing Link" between artist-friendly workflows and modern web development!


<br/>

# Needle Engine CHANGELOG

Source: https://github.com/needle-tools/needle-engine-support/releases
# Changelog
All notable changes to this package will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [4.4.0-beta] - 2025-03-28
- Add: `ObjectUtils.createPrimitive()` now supports scale as array   
    e.g. `ObjectUtils.createPrimitive("Cube", { scale: [1, .25, 1] } );`
- Add: Input `getGamepad(<index>)` function to query a connected gamepad. Example: `this.context.input.getGamepad()`
- Add: `lookAtScreenPoint()` function which allows 3D object to look at points in 2D screen coordinates (e.g. your mouse position).    
    **Example Component that makes the object look at the mouse**
    ```ts
    import { Behaviour, lookAtScreenPoint } from "@needle-tools/engine";

    export class LookAtMouse extends Behaviour {
      update() {
        lookAtScreenPoint(this.gameObject, this.context.input.mousePosition, this.context.mainCamera);
      }
    }
    ```
- Add: SyncedTransform does now also sync scale changes
- Add: Default environment lighting. If you don't configure any environment-image needle engine will now create a default scene to light your objects. Previously the scene was just black when the loaded model didn't contain any lighting information.
- Add: The `contactshadows` attribute now allows you to specify a factor for controlling the darkness/lightness. E.g. `<needle-engine contactshadows="1">` will make the shadows appear darker vs. `<needle-engine contactshadows=".2">` will make the contact shadow appear lighter.
- Fix: Implicit camera did not automatically set to skybox when using `background-image`. E.g. `<needle-engine background-image="<url>"> 
- Fix: Tonemapping falsely set tonemappingEsposure to undefined causing a black screen
- Fix: `background-color` attribute was not always applied. E.g. `<needle-engine background-color="#ff3333">`
- Change: License check aborts now faster instead of retrying when connection is actively refused
- Change: OrbitControls `autoTarget` does now automatically update the look at target after panning and not when rotating the camera (previously the target would be updated after any input but this resultet in undesireable behaviour when rotating around objects)
- Change: When no background-color is defined by either the loaded scene or by a `background-color` attribute then the default scene background color respects the user's accessibility setting for `prefer-dark` or `prefer-light`. This means that the background color will be set to a default light or dark value.

## [4.3.2] - 2025-03-20
#### Added
- Documentation for `Gizmos` API
- `this.context.time.fps` returning the FPS for the current frame (for a more stable FPS value `smoothedFps` can be used)

#### Removed
- Vite license plugin check does not support Node 16 anymore

#### Changes
- Improve Vite preload link injection into HTML head
- The PostprocessingManager component does now expose a `multisampling` property which is set to `"auto"` by default. By setting `multisampling` to a number it will force postprocessing to the configured samples.
- Downloading of `<needle-engine>` attributes for `skybox-image` and `environment-image` does now start earlier. Previously it would only start loading the HDRi or EXR images after the root glTF file was finished loading. Now it will start downloading earlier which improves the time until the scene is ready to be displayed.

#### Fixed
- Vite dependency-watcher plugin warning
- Vite license check plugin is now using the latest Needle CLI version
- Browser cache busting issue related to loading the root scene where range requests (e.g. when download urls without a file extension). This caused Chrome to remove files from the disc cache causing a re-download
- Three.js core postprocessing effects support
- Loading glTF files without any components (e.g. when directly downloaded from Sketchfab) was causing Needle Engine to falsely keep a reference to the loaded glTF structure. 
- Instancing issue where instancing did sometimes renderer wrong geometry when many objects where removed and added again and multiple different geometries were batched together. This was caused by a bug in the internal bucketing mechanism and has now been removed since it's not necessary anymore.

## [4.3.2-beta.5] - 2025-03-20
- Change: Improve Vite preload link injection into HTML head
- Fix: Vite dependency-watcher plugin warning

## [4.3.2-beta.4] - 2025-03-19
- Fix: Start loading earlier for `skybox-image` and `environment-image` when configured in `<needle-engine>` web component. Previously it would only start loading the HDRi or EXR images after the root glTF file was finished loading. Now it will start downloading earlier which improves the time until the scene is ready to be displayed.
- Fix: Update vite license check using the latest CLI version
- Fix: Issue related to loading the root scene where range requests (e.g. when download urls without a file extension) caused Chrome to remove previously full downloads to be removed from disc cache
- Remove: Vite license check does not support Node 16 anymore

## [4.3.2-beta.3] - 2025-03-18
- Fix: Support for three.js core postprocessing effects

## [4.3.2-beta.2] - 2025-03-18
- Add: Documentation for `Gizmos` API

## [4.3.2-beta.1] - 2025-03-17
- Fix: Bug when loading glTF files without any components where a reference to the loaded glTF structre was kept in memory causing the memory to not be freed

## [4.3.2-beta] - 2025-03-14
- Add: `this.context.time.fps` returning the FPS for the current frame (for a more stable FPS value `smoothedFps` can be used)
- Change: The PostprocessingManager component does now expose a `multisampling` property which is set to `"auto"` by default. By setting `multisampling` to a number it will force postprocessing to the configured samples.
- Fix: Issue where instancing with multiple different geometries being batched together did sometimes be cause wrong rendering when many objects where removed and added again. This was caused by a bug in the internal bucketing mechanism which is not necessary anymore with the new BatchedMesh version.

## [4.3.1] - 2025-03-14
- Add: Expose lifecylcle hooks in `Needle` global scope for usage in commonjs or without a bundler. For example this allows to subscribe to the update event with `Needle.onUpdate(ctx => console.log(ctx.time.time))`.
- Fix: OrbitControls micro-movement after a pointer event was already used. For example previously when using DragControls and starting to drag an object the camera would still move slightly before stopping during drag. This is not the case anymore now and the camera does not move anymore during interaction with other objects.
- Fix: Canvas UI render settings not being applied in one rare case causing a worldspace image not being set to double sided rendering.

## [4.3.0] - 2025-03-12
- Bump version to 4.3.0

## [4.3.0-alpha.6] - 2025-03-11
- Change: Postprocessing effects in shared volume (when exported from Unity) are now added to the Volume gameObject during initialization
- Fix: improve Rigidbody physics poststep / smoothed velocity

## [4.3.0-alpha.5] - 2025-03-06
- Add: more jsdoc comments to `Graphics.textureToCanvas` and `Mathf` methods
- Change: AudioSource tries to get AudioListener from it's own object first before it checks camera and scene
- Fix: SpriteRenderer issue where runtime instantiate and setting Sprite would in some cases not update the texture on all instances
- Fix: Issue in `Graphics.copyTexture` where the blit material was not yet created

## [4.3.0-alpha.4] - 2025-03-06
- Revert last AudioSource change

## [4.3.0-alpha.3] - 2025-03-06
- Add: WebARSessionRoot `arScale` can now be changed while in AR to modify or reset the scale.
- Change: AudioSource now creates it's own AudioListener instead of using one from the camera. 

## [4.3.0-alpha.1] - 2025-03-03
- Add: More API documentation for various core components

## [4.3.0-alpha] - 2025-02-28
- Add: SceneSwitcher `sceneLoadingStart` and `sceneLoadingProgress` events
- Add: AssetReference `urlName` property
- Fix: SceneSwitcher preloading if configured to not load scene at startup 
- Fix: UI issue where text components were not correctly handled by the EventSystem for the `hasActiveUI` property

## [4.2.5] - 2025-02-27
- Fix: keep FBX vertex color assignment when postprocessing materials
- Fix: ContactShadow flicker when point materials are in the scene
- Fix: OrbitControls keep `autoTarget` enabled in `fitCamera`
- Fix: Static `BoxCollider.add` now correctly calculates bounding box 
- Fix: InputField `onValueChanged` event is now invoked after the new value has been assigned
- Change: `getBoundingBox` now also takes a single object as well as an array of objects
- Change: DropListener now exposes `DropListenerOnDropArguments` type

## [4.2.4] - 2025-02-21
- Add: OrbitControls set to `autoTarget` now automatically updates rotation target in onPointerUp
- Fix: USDZ add safeguard against potential issue in bone sorting
- Fix: USDZ remove extra check for bone type that might prevent bone structure ordering
- Fix: Static method for `BoxCollider.add` not correctly calculating object bounds when object is rotated 
- Fix: Ignore fullscreen plane for WebXR auto center
- Fix: Issue in FBXLoader where loading FBX files with out-of-bounds material assignments lead to incorrect geometry groups and subsequent errors
- Change: Improve input typings for `isKeyDown`, `isKeyUp` and `isKeyPressed`

## [4.2.3] - 2025-02-20
- Add: InputField setter for `text` 
- Fix: OrbitControls setCameraAndLookTarget
- Fix: Issue where updating KTX transcoder was not being correctly applied for offline usage 

## [4.2.2] - 2025-02-18
- Fix: WebXRImageTracking iOS size calculation due to change by Apple
- Fix: USDZ AudioSource not generating code for `playOnAwake` anymore
- Fix: Lightmap and environment ligthing not working correctly anymore due to change in three v163 
- Change: EventSystem simplification - it is now always created once in scene root, this removes the requirement of ObjectRaycasters in the scene which simplifies the usage of component input event methods like `onPointerClick` (which previously required users to ensure there are ObjectRaycasters in the parent hierarchy)

## [4.2.0] - 2025-02-17
- Change: remove MXInk fallback codepath for pre-release OS versions
- Change: don't request "hand-tracking" on VisionOS devices unless explicit custom hand models are requested
- Fix: occluder generation in Plane/Mesh tracking should not be enforced when we have a data template

## [4.1.1] - 2025-02-14
- Fix: ParticleSystem regression where particles using a transparent PBR shader are not being rendered as transparent

## [4.1.0] - 2025-02-13
- Fix: Issue in WebXR component where disabling `usePlacementReticle` would not apply other settings to already existing WebARSessionRoot components in the scene

## [4.1.0-beta.9] - 2025-02-12
- Add: `Object3D.hideFlags` property and ContactShadows sets generated shadow object to `DontExport` to be excluded in runtime glTF export
- Fix: ContactShadows `auto()` does now check if the scene already contains contact shadows 

## [4.1.0-beta.8] - 2025-02-12
- Fix: Handle `<needle-menu>` web component being present anywhere in the document already
- Fix: Allow nextjs based projects to be deployed to Needle Cloud
- Change: `instantiate()` can now take an AssetReference as an argument as well
- Change: Improve `context.input.addEventListener` function typing 

## [4.1.0-beta.6] - 2025-02-05
- Fix: issue with `parseSync` for loading a glTF file is a file path

## [4.1.0-beta.5] - 2025-02-04
- Fix: Physics issue with mesh colliders where colliders were sometimes created with the wrong size depending on the users network speed. This was due to the change of lazily loading the physics engine in Needle Engine 4.
- Fix: Add workaround for a regression in the particle system where PBR materials would sometimes render with a wrong texture. [Issue](https://github.com/Alchemist0823/three.quarks/issues/101)
- Fix: ParticleSystem gradients set to `Random Color` 
- Fix: import WebGLNodes to ensure nodes actually run
- Change: Physics engine `addMeshCollider` scale argument is now optional and treated as scale applied to the object's world scale when creating the collider.

## [4.1.0-beta.4] - 2025-02-03
- Add: OrbitControls support for orthographic camera
- Fix: Orthographic camera aspect ratio

## [4.1.0-beta.3] - 2025-01-30
- Fix: issue where networked values with the `@syncField` decorator would not be updated correctly on instances spawned at runtime (e.g. via PlayerSync).

## [4.1.0-beta.2] - 2025-01-29
- Fix: VideoPlayer screenspace mode colorspace 
- Fix: XRRig `setAsActiveRig()` now ensures the given rig has the highest priority. Previously it was possibly immediately overidden again if another XRRig with a higher priority was active in the scene.
- Change: SpatialTrigger EventList events are now invoked without additional parameters. They did previously take a reference to the SpatialTriggerReceiver and active SpatialTrigger. If this information is important the SpatialTriggerReceiver can be saved when subscribing to the event and the trigger can be accessed using the `triggerReceiver.currentIntersected` array.
- Change: TransformGizmo now exposes the underlying three.js TransformGizmo via a new `control` property.
- Change: GroundProjection now uses the `scene.background` instead of `scene.environment`

## [4.1.0-beta.1] - 2025-01-29
- Fix: ParticleSystem custom behaviours
- Fix: OrbitControls not keeping the initial look direction if `autoTarget` was off
- Fix: Invalid Rigidbody warning caused by async physics engine loading change
- Fix: SharpeningEffect
- Update gltf-progressive including a fix for loading texture LOD 0 (the highest resolution)
- Update three.quarks dependency to 0.15.6

## [4.1.0-beta] - 2025-01-24
- Promote to beta

## [4.1.0-alpha.14] - 2025-01-24
- Fix: glsl shader error at linear to sRGB conversion

## [4.1.0-alpha.13] - 2025-01-23
- Update three.js dependency to fix issue in WebXRManager

## [4.1.0-alpha.12] - 2025-01-21
- Change: Much improved initial loading speed making Needle Engine websites load up to 1000 ms faster.  
  Improved Needle Engine website lighthouse scores for all Vite based projects.   
  Improved bundle chunking and loading for better lighthouse scores.    
  Core bundles and entrypoint glTF files are not loaded as soon as the web document is ready.
- Change: Needle Engine does now lazily import dependencies for `rapier` and `postprocessing` to reduce download and startup size.   
  This means that projects that don't use any collider or rigidbody components will not load the physics and postprocessing modules. Modules can be loaded at any time once they're need by e.g. components in the scene.

## [4.1.0-alpha.11] - 2025-01-17
- Remove: legacy Flycontrols component
- Fix: MeshCollider contact normal direction

## [4.1.0-alpha.10] - 2025-01-17
- Fix: Instanced meshes bug where sometimes an vertex count increase due to mesh compression was not handled correctly

## [4.1.0-alpha.9] - 2025-01-15
- Fix: Better AgX and Neutral tonemapping

## [4.1.0-alpha.8] - 2025-01-14
- Update three.js to r169
- Update dependencies: three-mesh-bvh to 0.8.3, three-quarks to 0.15.5, postprocessing to 6.36.6, n8ao to 1.9.3
- Fix: AR screenshot bug where the second screenshot would not look correct
- Fix: nextjs support

## [4.1.0-alpha.5] - 2025-01-13
- Fix: Issue where MeshCollider with InterleavedBufferAttributes were not created correctly
- Fix: multi-object mesh collider handling for Groups and add warning for remaining case that are not supported
- Fix: transparency trimming for screenshot should happen before screenshot is returned
- Change: Reduced mesh LOD vertex reduction to improve quality of lowest LOD 
- Update build pipeline to latest 2.7 alpha
- Update rapier dependency to 0.14.0

## [4.1.0-alpha.4] - 2025-01-10
- Add: SceneSwitcher methods for `unload()` and `reload()` the currently loaded scene

## [4.1.0-alpha.1] - 2025-01-09
- Add: Support for loading KTX2 textures as `environment-image` and `background-image` (e.g. `<needle-engine background-image="my-skybox.ktx2">`)
- Fix: Issue where raycasting was not using acceleration structures and instead falling back to slow raycasting. This release greatly improves raycasting performance

## [4.1.0-alpha] - 2025-01-08
- Add: Screenshot support for AR:   
  To use you need to request the `camera-access` XR feature when starting a new XR session.   
  For that you can *either* simply add the *WebARCameraBackground* component to your scene   
  or add the feature in the `NeedleXRSession.onSessionRequestStart` event hook. 

## [4.0.4-alpha] - 2025-01-02
- Fix: Detecting OBJ files starting with `mtllib` or content-type header `model/obj`
- Fix: Issue where `screenshot()` method with custom camera and postprocessing didn't work
- Update gltf-progressive package to load higher texture LODs by default

## [4.0.3-alpha] - 2014-12-20
- Add: OBJLoader now automatically loads mtl files
- Add: OBJLoader materials are now fixed like FBX materials 

## [4.0.2-alpha] - 2024-12-12
- Add: Support for custom scripts on WebXR custom hands
- Fix: OrbitControls should not listen to window.body for key events

## [4.0.1-alpha] - 2024-12-11
- Fix: Objects spawned with `syncedInstantiate` should automatically be removed when disconnecting from the networking room
- Fix: Issue where objects rendered with instancing enabled were not visible in screenshots

## [4.0.0-alpha] - 2024-12-09
- Add: Builds can now be compressed on Needle Cloud instead of locally. Set the `NEEDLE_CLOUD_TOKEN` environment variable to your Cloud access token to enable this. This allows running compression in CI build pipelines, for example in a Github Action. Needle Cloud access tokens can be obtained on https://cloud.needle.tools/team.
- Add: `DropListener` now optionally supports networking when `allowNetworking` is on. Dropped files will be network-synced if the file size does not exceed a certain size (50 MB for projects on paid plans; 5 MB for projects free plans).
- Fix: Previously, when changing a `@syncField` property, only the last changed property was synchronized to the server. This is now fixed. Note that this is a **breaking change**: previously networked server state with the `@syncField` attribute in Needle Engine 3.x projects will not be loaded in Needle Engine 4.0.
- Change: `EventList` now takes a generic type code hints when using `myEventList.addEventListener(...)` e.g. `new EventList<string>()` 
- Change: The default networking server is now located at `networking.needle.tools`
- Change: The `DropListener` API has changed
- Change: OrbitControls `useKeys` is now false by default since the underlying three.js OrbitControls incorrectly capture ALL keyboard events on the page.
- Change: Loading speed improvements and improved Lighthouse score
- Change: md5 hashing results are now S3-compatible

## [3.51.1] - 2024-12-09
- Fix: DragControls snap to surface setting should use world normal
- Fix: Needle Menu missing focus-visible for overflow menu button
- Fix: Needle Menu :focus-visible on menu elements barely being visible
- Fix: edge case where we still showed the QR code when only the WebXR component is on and we're on mobile
- Fix: detect URL content format using "Content-Type" header as well
- Fix: double quotes in USDZ displayName need to be escaped
- Fix: don't fetch when we already know fetch won't work due to cross-origin isolation
- Change: for QuickLook, use "View in AR" instead of "Download for AR" even for cases where we're not sure if the device actually supports QuickLook – rel="ar" is not reliable
- Change: disable Needle logo being clickable during loading because users click it accidentally and then they're lost
- Docs: add @group Component to components since TypeDoc doesn't do inheritance as expected

## [3.51.0] - 2024-11-28
- Add: API docs for networking methods
- Add: USDZ export automatically creates "hold" animations now to prevent animation snapping
- Add: USDZ export now supports ShadowCatcher component and materials
- Add: USDZ export now contains more meta-info about exported animations
- Add: Log warnings on USDZ export for overlapping animation roots so hierarchy can be manually adjusted
- Add: Use ?debugusdzbehaviours to log more information and Behaviour Graphs during USDZ export. A graph in mermaid format will be logged into the console, which can be pasted into https://massive-mermaid.glitch.me or other viewers. 
- Fix: USDZ animation rounding error edge cases for long-running animations (>20.000 frames)
- Fix: USDZ export only adds physics extension now if required to work around QuickLook issue (FB15925487) for animated objects with physics components
- Fix: USDZ exporter adds rest poses properly again
- Fix: Developer console is now more robust against network errors, e.g. offline PWA usage
- Fix: Canvas parent element size now matches the canvas size

## [3.50.0-beta] - 2024-10-28
- Add: PWA plugin now works with gzip compression
- Add: Asynchronous overload for `screenshot2` method that returns a `Promise<Blob>`
- Add: USDZExporter supports `animator.minMaxOffsetNormalized` and `animation.minMaxOffsetNormalized` for QuickLook now
- Add: USDZExporter vertex color export. Use unlit white materials with only vertex colors to export as `displayColor` without a material. This is supported on iOS 18+ and visionOS 2+.
- Fix: automatically instanced/batched meshes are not exported twice in USDZ anymore.
- Change: "Open in QuickLook" is now "View in AR" to be more consistent
- Change: Deprecate device utilities outside the `DeviceUtilities` namespace, use the methods directly from the namespace instead (e.g. `DeviceUtilities.isSafari()`)
- Change: legacy `Object3D.transform` wrapper that returns itself

## [3.49.0-beta] - 2024-10-21
- Add: More jsdoc code API documentation
- Add: `OrbitControls` can now lerp to a target field of view
- Add: Timeline animation tracks now can have 0..1 weights
- Add: `getBoundingBox` can now optionally respect a layer mask for determining which objects to include
- Fix: Regression with animations not playing on instantiated objects
- Fix: `DeleteBox` uses `getBoundingBox` now instead of a custom implementation
- Fix: `Duplicatable` now correctly checks for its target object already being deleted
- Fix: vite plugins use vite 5.x API (and are still compatible with 4.x)
- Fix: MX Ink support for QuestOS v71+
- Fix: Some logs where incorrectly not behind debug flags
- Fix: Don't export cameras in USDZ when requesting QuickLook-compatible assets
- Fix: Exclude childs of `ContactShadows` and `GroundProjectedEnv` from USDZ export
- Fix: Custom hand models should not automatically become occluders in Passthrough XR
- Fix: Avoid potential exception in `GameObject.getAllComponents(...)`
- Fix: Prevent exception in `DragControls` when dragged object or controller disappear mid-drag
- Fix: Improve `EventList` usage from pure JS
- Fix: Regression in `OrbitControls.autoFit` that resulted in a wrong fit in some cases
- Fix: Improve order of batch validation for automatic instancing when some meshes have different attributes
- Change: Needle Menu styles are more consistent now
- Change: Change casing on `isiPad()`, deprecate `isIPad()`

## [3.48.3] - 2024-09-30
- Add: `NeedleXRSession.start("ar")` support which will start `immersive-ar` on supported devices or export an interactive USDZ on iOS devices
- Add: nextjs alias plugin
- Fix: Voip on iOS with connected bluetooth device should select the bluetooth device  

## [3.48.1] - 2024-09-24
- Add: `<needle-engine>` attributes for `background-color`, `background-blurriness` and `background-image`
- Fix: Loading of OBJ files exported from ZBrush
- Fix: Loading of FBX files with empty vertex color attributes
- Fix: Unregister mixer in Animation component 
- Fix: Needle Menu CSS improvements when using custom html elements inside the menu
- Fix: Needle Menu top positioning now takes CSS safe area into account
- Fix: Balloon message styles adjusted
- Fix: ContactShadows auto fit calculation
- Change: Exit AR button style aligned with Needle Menu

## [3.48.0] - 2024-09-19
- Add: Logitech MX Ink controller support
- Add: Quest menu button now shows Needle Menu in immersive-ar and immersive-vr
- Fix: Improved USDZ code generation regarding transformations/matrices
- Fix: USDZ multi-channel attribute export
- Fix: Minor jsdoc comment error
- Fix: Postprocessing Volume `removeEffect` does now also remove effects from assigned `sharedProfile`
- Internal: Warn if Rigidbody physics object could not be created due to a missing collider

## [3.47.8] - 2024-09-10
- Fix: LookAt orientation fixed when both `copyRotation` and `keepUp` are active
  - If you have both options enabled and rely on the previous behaviour, toggle `invertForward` to get the same result as before. 
- Fix: USDZExporter correctly applying `arScale` again
- Fix: Issue with Needle Menu buttons preventing AR or VR to start in certain cases/devices 
- Fix: BatchedMesh/Instancing error after adding/removing many instances and growing the batched mesh buffers where internal state was not properly reset causing an error
- Change: Make Needle Menu text not selectable
- Change:`NeedleXRSession.start("immersive-ar")` now exports to USDZ when called on iOS (experimental) 
- Change: Needle Menu foldout is now closed when entering XR
- Change: Remove `Quit XR` button for screen based AR experiences (in favor of the X icon at the top right corner)
- Change: Needle Menu foldout button has now an larger click area to avoid accidental clicks on the Needle logo
- Change: LookAt `copyTargetRotation` is now disabled automatically during VR/passthrough AR session

## [3.47.7] - 2024-09-04
- Fix: Support loading FBX files in ASCII format
- Fix: Support loading OBJ files starting with `# Alias OBJ`
- Change: Needle Menu buttons height adjusted for mobile

## [3.47.6] - 2024-09-02
- Fix: USDZ animation interpolation. We're now using timesamples for `translation`, `orientation` and `scale` separately. This fixes an issue with animation optimization (optimization removing redundant keyframes) to just 2 keyframes where matrix timesamples would produce unexpected interpolations
- Change: inline mesh bvh worker

## [3.47.6-beta] - 2024-09-02
- Add: `Mathf.random()` can now also accept an array to randomly select an element
- Add: AudioSource `pitch` property
- Add: SpriteRenderer `addSprite` method
- Add: SpriteRenderer `texture` setter to change the texture of the currently rendered sprite
- Fix: SpriteRenderer `sprite` setter
- Fix: OrbitControls double click to focus should not focus if the click was already used (e.g. by a button)
- Fix: XR `screenshot()` support
- Fix: DragControls now work with physical objects again e.g. Rigidbodies that react to gravity

## [3.47.5-beta.1] - 2024-08-28
- Add: `screenshot` option to output a texture
- Fix: `screenshot` method when using a camera with a RenderTexture assigned
- Fix: `ObjectUtils.createPrimitive` option linting
- Change: Addressables `instantiate` method now returns Object3D type

## [3.47.5-beta] - 2024-08-27
- Fix: Shader warmup when loading glTF file containing *only* materials (no scenes)
- Fix: OrbitControls `autoTarget` option causing `setLookTargetPosition` to be overriden during the first frame 
- Fix: Compressed RenderTexture not being updated on objects in the scene
- Fix: Physics mesh BVH generation on worker caused raycasts during processing to not work until the BVH was ready 

## [3.47.4-beta.3] - 2024-08-26
- Fix: three.js changing environment texture while in XR 
- Fix: three.js OrbitControls zoom damping
- Fix: GroundProjectedEnv error when setting `height` as part of init properties as part of `addComponent(..., { height: 4 })`
- Fix: OrbitControls `minZoom` and `maxZoom` should be applied during update
- Change: Camera `backgroundBlurriness` and `backgroundIntensity` should be undefined by default

## [3.47.4-beta.1] - 2024-08-24
- Add: Support loading of glTF files in `<needle-engine src="...">` that don't contain any objects/scenes but just materials. These glTF files will be rendered with a shaderball
- Fix: Regression in ParticleSystem trail rendering where `mode` was not set correctly
- Change: OrbitControls `fitCamera` can now also take a single object as a first parameter

## [3.47.4-beta] - 2024-08-17
- Add: OrbitControls `fitCamera` method overload which can directly take an options parameter. E.g. it can be invoked with `fitCamera({ immediate: false })`
- Fix: Lifecycle hooks like `onStart` and `onInitialized` are now properly called again for new subscribers
- Fix: Regression in raycasting with multi-material objects
- Fix: `<needle-engine>` canvas highlighting with touch on iOS
- Change: The WebGL context is now automatically restored when lost

## [3.47.3-beta.2] - 2024-08-14
- Fix: Issue where automatic camera change stopped working
- Fix: `screenshot` should update the camera aspect ratio before rendering 
- Fix: Physics raycasts do now run basic geometry validation 

## [3.47.3-beta] - 2024-08-13
- Add: Improved ParticleSystem MinMaxCurve and MinMaxGradient types with utility methods (`constant`, `betweenTwoColors` / `setConstant`, `setMinMaxConstant`, `setCurve`) 
- Add: preliminary support for MX Ink pens
- Add: NeedleXRController `pinchPosition` getter for hand tracking
- Add: SceneSwitcher add `sceneLoaded` Eventlist
- Add: DragControls static `CurrentlySelected` getter to access all currently active DragControls components
- Fix: Gizmo label offset
- Fix: XRControllerFollow on VisionOS with hand tracking
- Fix: Vision OS depth buffer workaround (FB14720123)
- Fix: emulate grip space for hands that don't have grip space
- Fix: `screenshot2` should use passed in camera
- Fix: nextjs plugin fix for error caused by mesh bvh worker integration
- Fix: `input.mouseDoubleClick` getter
- Fix: EventList invocation with custom arguments (e.g. `myEvent.invoke("test")`)

## [3.47.2-beta.3] - 2024-08-08
- Fix: Properly resolve EventList calls when using `instantiate` to the new instances
- Fix: WebXR simulator hand gesture calculation
- Fix: WebXR desktop preview with postprocessing enabled
- Fix: WebXR input `click` detection 

## [3.47.2-beta.2] - 2024-08-06
- Add: Type definitions for Object3D Needle Engine extension methods like `addComponent` or `worldPosition`
- Add: ObjectUtils `createText`
- Add: static `BoxCollider.add` method
- Add: Experimental util method `PlayerSync.setupFrom` to easily setup networked player representations 
- Add: ContactShadows `minSize` property
- Fix: ShadowCatcher should set `receiveShadow` to true
- Fix: Animation `play` should restart the animation if it's at the end
- Fix: ContactShadows should ignore Line materials
- Fix: SyncedRoom component is easier to setup from code
- Change: Physics collider center x should not be flipped. Instead the exporters need to ensure the correct space  

## [3.47.1-beta] - 2024-08-05
- Add: Voip microphone button option to allow users to mute and unmute themselves
- Add: More jsdoc documentation
- Fix: Improved audio and voicechat streams when using the Voip component to more reliably play audio and connect to all users in the room
- Fix: Prevent browsers translation of HTML icons
- Fix: Issue with Application audio playback permissions check
- Fix: SpriteRenderer setting `renderOrder` must be rounded to an integer

## [3.47.0-beta.3] - 2024-08-01
- Add: NeedleXRController `emitPointerDown`, `emitPointerUp` and `emitPointerMove` properties to control if the controller should emit pointer events to the Needle Engine Input System
- Add: NeedleXRController `pointerMoveDistanceThreshold` and `pointerMoveAngleThreshold` for changing when controller `pointermove` events are emitted. These values can be set to 0 to emit events every frame or larger values to reduce the number of events emitted.
- Add: Support to disable SpatialGrabRaycaster by setting the static property `SpatialGrabRaycaster.allow = false`
- Fix: Vite issue where mesh bvh worker was not found in local dev server
- Fix: Mesh BVH should not raycast on meshes that don't have a position attribute
- Change: EventSystem now respects used events. This means if you subscribe to `pointerdown/pointermove/pointerup` with the queue set to a negative value and call `event.use()` or `event.preventDefault()` the EventSystem will ignore the event.

## [3.47.0-beta] - 2024-07-31
- Add: accelerated raycasting using three-mesh-bvh. All calls to `physics.raycast()` now use a spatial grid solution under the hood to improve raycasting performance. (This can be disabled in the raycasting options by setting `useAcceleratedRaycast: false`)
- Add: `physics.engine.raycast` methods now have an option to ignore the `ignoreRaycastLayer` on the Object3D (the three.js layer 2 is used to exclude objects from raycasting. This means setting `layers.set(2)` is equivalent to "Do not raycast on this object")
- Minor UI performance improvements

## [3.46.1-beta.4] - 2024-07-30
- Add: SpriteRenderer `toneMapped` option
- Fix: minor iOS color fix in Needle Menu CSS
- Change: SyncedRoom auto-generated room name is now numbers only by default

## [3.46.1-beta.3] - 2024-07-29
- Fix: Voip should stop audio sending and receiving when user disconnects from a networked room
- Fix: WebXR avatar head being offset when user is not centered in rig space

## [3.46.1-beta.2] - 2024-07-29
- Fix: WebXR teleport and rotation now takes user position in rig space into account. This means when teleporting the user is now placed at the expected ray target point and when rotating the user is rotated around himself as expected (and the position stays the same)
- Fix: Animation component `fadeDuration` didn't fade out previously playing animations but instead stop immediately

## [3.46.1-beta.1] - 2024-07-29
- WebXR performance improvements
- Add: Needle Menu foldout submenu for smaller screen sizes. The compact Needle Menu does now have a submenu which can be opened by clicking the 3 dots button at the right side. This improves the menu layout for smaller screen sizes.
- Fix: ContactShadows performance improvements
- Fix: Gizmo label performance improvements

## [3.46.1-beta] - 2024-07-26
- Add: `isMacOS` utility method
- Improve WebXR controller and hand raycasting and line rendering. Hide rays while the primary pointer i active.
- Improve DragControls performance
- Fix: Animation `play` should resume paused animations instead of restarting
- Fix: PWA plugin now automatically disables gzip compression. Improved error message if there's missing configuration
- Fix: SyncedRoom `joinRoom` button not creating a new room if the room url parameter was present but empty
- Change: Better error message for circular imports causing scripts to not work 
- Change: On OSX and MacOS use power-preference `default` for improved performance ([webkit issue](https://bugs.webkit.org/show_bug.cgi?id=276959))

## [3.46.0-beta.5] - 2024-07-23
- Add: WebXR teleport using right hand pinch
- Add: WebXR `Quit XR` button
- Add: SceneSwitcher `useSceneBackground` option to apply background skybox from loaded scene (if it has any)
- Add: SceneSwitcher option to add Needle menu buttons
- Add: `Gizmos.DrawCircle`
- Fix: issue where `depth-sensing` would cause camera near and far planes to have invalid values causing the scene to not render properly anymore when exiting AR
- Fix: DropListener should not prevent propagation. Otherwise e.g. RemoteSkybox and DropListener would not work together
- Fix: DropListener `placeOnHit` should not raycast on just dropped object
- Fix: issue where onPointerEnter is only invoked once if onPointerExit isnt declared
- Fix: WebXRController hits rendering on UI elements
- Fix: WebXRController rays are always visible when enabled but with a low opacity if no object was hit
- Change: WebXR `autoPlace` does now only place on flat surfaces with a small time threshold
- Change: DragControls `SnapToSurface` now uses `DynamicViewAngle` if no object was hit
- Change: Needle Menu active and focus button CSS

## [3.46.0-beta.1] - 2024-07-23
- Add: MeshTracking (WebXRPlaneTracking) is by default now generating occluder meshes for detected planes and meshes
- Fix: Animation `playOnAwake` regression
- Fix: ContactShadows ignoring GroundProjectedEnv sphere, gizmos and depth only objects (e.g. occluder spheres from mesh tracking)
- Fix: Ensure Sprites when exporting to USDZ 
- Change: Settings on WebARSessionRoot are now all moved to the WebXR component
- Update three.js dependency to 0.162.6

## [3.46.0-beta] - 2024-07-22
- WebXR performance improvements for immersive-ar sessions as well as controller and hand tracking
- Add: GroundProjection can now blend with the AR environment using the `blending` property
- Change: WebXR controller hits now only use object bounds

## [3.45.2-beta.5] - 2024-07-22
- Add: AudioSource properties for `time`, `time01` (normalized time), `duration`
- Add: Animation properties for `time` and method for `pause()`
- Add: WebXRPlaneTracking `occluder` property to automatically generate occluder meshes for detected meshes
- Change: Hide GroundProjection in immersive-ar
- Update three.js dependency to 0.162.5

## [3.45.2-beta.4] - 2024-07-19
- Fix: DragControls `Snap To Surface` when starting to drag
- Change: Minimal increase of close distance for SpatialGrab in WebXR

## [3.45.2-beta.3] - 2024-07-18
- Add: Support for immersive-ar `unbounded` session for unlimited passthrough experiences (enabled by default).   
  This is currently an experimental feature in the Quest browser. To enable it go to `chrome://flags` and check `WebXR experiments`, then restart the Browser. The next time you start an immersive-ar session on quest the WebXR experience will run without guardian boundaries.
- Add: NEPointerEvent `isSpatial` getter to easily determine if an input event was generated by a spatial device
- Fix: Raycast handle null or empty objects in targets array

## [3.45.2-beta.1] - 2024-07-17
- Add: Object `static:true` does now disable `matrixAutoUpdate` 
- Fix: Animation component `play()` issue where previous animations were not always stopped/faded out
- Fix: SceneSwitcher regression causing scenes to not be unloaded
- Change: pointer events are now captured and continue to work when they hover over other HTML elements (if started in the 3D scene)

## [3.45.1-beta.7] - 2024-07-16
- Add: vite plugin that ensures that the npm folder exists
- Fix: Issue where completely empty scene without explicit XR rig caused invalid placement/rendering in VR
- Change: Disable touch-action on `<needle-engine>` element

## [3.45.1-beta.5] - 2024-07-16
- Update gltf-progressive to version 1.2.5

## [3.45.1-beta.3] - 2024-07-15
- Add: GroundProjection blurriness set from `scene.backgroundBlurriness`
- Fix: `getTempVector` and `getTempQuaternion` set to 0,0,0 by default to return consistent values when called without parameters

## [3.45.1-beta.1] - 2024-07-13
- Add: GroundProjection `autoFit` option
- Fix: OrbitControls re-applying autoFit again when enabled/disabled
- Fix: `getBoundingBox` should ignore gizmos

## [3.45.1-beta] - 2024-07-12
- Add: Camera `backgroundRotation` property
- Add: GroundProjection environment rotation using `scene.backgroundRotation`
- Fix: XRControllerModel `createHandModels` not working when `createControllerModels` was disabled
- Change: WebXR avatar does now hide local hand models when XRControllerModel models or hands are enabled

## [3.45.0-beta.1] - 2024-07-11
- Add: The WebXR component now uses a static avatar from the CDN if none is assigned

## [3.45.0-beta] - 2024-07-11
- Add: XRControllerFollow does now reset the object to the original pose after XR
- Fix: XRControllerFollow does now take original object scale into account
- Fix: OrbitControls autoFit now takes GroundProjection scale into account for the camera far plane 
- Change: ContactShadows do now apply scale and transform to a child object so it does not modify the transform anymore of the object it was added to
- Change: GroundProjectionEnv default height to 3
- change: Increase GroundProjectionEnv resolution to 128 for smoother edges
- Change: ChangeMaterialOnClick does now change mouse cursor on hover

## [3.44.6] - 2024-07-10
- Add: The default XR rig is now automatically placed to view the scene if no explicit XRRig component is found in the scene 
- Add: XRControllerMovement does now allow teleportation on the current rig ground plane (if no object is hit when trying to teleport)
- Fix: XRController ray rendering should respect rig scale if the ray doesnt hit any object in the scene
- Fix: rare issue in input calculating world position from screenspace input
- Change: Voip component `autoConnect` default changed to true

## [3.44.5] - 2024-07-10
- Add: `window[Needle].NeedleXRSession`
- Change: WebARSessionRoot reticle without depth test for better use with depth sensing
- Change: VR default movement speed increased slightly from 1 to 1.5
- Change: VR default hit visualization adjusted to improve raycasting.

## [3.44.4] - 2024-07-09
- Fix: SceneSwitcher can now activate and deactivate objects in the scene
- Fix: `XRControllerFollow` takes rig scale into account
- Change: WebXR controllers do not raycast the scene for hit visualization anymore by default. To use add `XRControllerMovement` and enable `showHits`

## [3.44.3] - 2024-07-05
- Add: TonemappingEffect `exposure` setting to control tonemapping exposure
- Fix: Animator transition for negative timescale

## [3.44.1] - 2024-07-05
- Add: `instantiate` can now be invoked to clone objects without components
- Fix: OrbitControls `autoTarget` and `autoFit` at the start of a scene
- Fix: KTX2Loader being created both by needle engine and gltf-progressive resulting in a warning
- Change: DragControls `SnapToSurface` is more snappy
- Change: DragControls `SnapToSurface` drag start behaviour improved

## [3.44.0] - 2024-07-04
- Add: `syncInstantiate(object, { deleteOnDisconnect:boolean })` and `IModel { deleteOnDisconnect:boolean }` option to delete a networked instance in the networking storage when a user disconnects. This requires the networking server to use `@needle-tools/networking@2.x` (instead of `@needle-tools/needle-tiny-networking-ws`)
- Fix: Improve `PlayerSync` networking, use `deleteOnDisconnect` 
- Fix: Update draco decoader to 1.5.7
- Fix: SharpeningEffect effect order
- Fix: SharpeningEffect alpha handling
- Fix: WebXR MeshTracking plane and mesh normals
- Fix: ChangeMaterialOnClick resolving of assigned material in certain cases
- Fix: Duplicatable + DragControls not properly networking on first duplication in cases where multiple DragControls components are nested 

## [3.44.0-beta.2] - 2024-07-02
- Add: Expose `AnimationUtils`
- Fix: Dont apply reflection probes to unlit materials
- Fix: FBX loading can now handle multimaterial objects
- Fix: WebXR networked Avatar head position
- Fix: Needle Menu layout update is now enforced immediately when options change
- Fix: Initialize Postprocessing effects parameter when created from code

## [3.44.0-beta.1] - 2024-07-01
- Add: support to use file extension for determining which loader to use to save initial header fetch 
- Add: Tuned AGX tonemapping
- Fix: Sharpening effect causing color overflow
- Fix: `screenshot()` breaking transparent rendering

## [3.43.0-beta.1] - 2024-06-29
- Add: `environment-image` magic names now use a lower-resolution version of the HDR/EXR image if the texture is just used for lighting
- Fix: Toggle Volume component `enable` should add and remove the effects
- Fix: Remove leftover log in `<needle-button>` 

## [3.43.0-beta] - 2024-06-28
- Add: Postprocessing Sharpening effect
- Add: PostprocessingManager `addEffect` and `removeEffect` API 
- Fix: Error in spatial menu caused by slots
- Fix: Export of WebXRButtonFactory type

## [3.42.0-beta] - 2024-06-27
- Add: `<needle-menu>` slot support
- Add: `<needle-button>` with support for `ar`, `vr` and `quicklook`. For example: `<needle-button ar>` creates a Needle VR button
- Change: default tonemapping is now tuned AGX

## [3.41.2-beta.3] - 2024-06-27
- Add: support to use file extension for determining which loader to use to save initial header fetch 
- Fix: Patched neutral tonemapping [issue](https://github.com/google/model-viewer/issues/4825)

## [3.41.2-beta] - 2024-06-25
- Add: DropListener can now take `dropArea` object to define a section of the scene that would accept the dropped file.
- Add: DropListener support to fit a mesh into a defined volume (e.g. 1x1x1)
- Add: SyncedRoom option to create a share button for the view only room URL
- Fix: DragControls `SnapToSurface` option when dragged object is not a mesh / the mesh is in the child hierarchy

## [3.41.1-alpha.1] - 2024-06-24
- Add: `<needle-engine tone-mapping>` attribute. Possible values are `none`, `linear`, `neutral` or `agx`.
- Change: Camera fitting near and far planes are now tighter to reduce z-fighting in some cases

## [3.41.0-alpha.5] - 2024-06-22
- Add: `<needle-engine>` web component attributes `environment-image` and `skybox-image` can now be set to presets values: "studio", "blurred-skybox", "quicklook" or "quicklook-ar" 
- Fix: ContactShadows should capture objects on all layers
- Fix: Issue in LOD gizmo debug rendering 
- Change: Use smaller icons font

## [3.41.0-alpha.4] - 2024-06-19
- Add: AnimationRegistry, accessible via `this.context.animations`
- Add: static AnimationUtils class
- Add: `TestSceneUtils.createComparisonScene` for quickly spinning up test scenes
- Add: Menu `appendChild` can now create a button from a structured json object 
- Add: New `exportAsGLTF` function
- Add: `<needle-engine contactshadows>` attribute
- Add: Lifecycle hooks like `onStart` or `onUpdate` can now be configured to only run once using the options argument. For example: `onBeforeRender(ctx => { ... },  { once: true })`
- Fix: Postprocessing where Tonemapping was not being applied in some cases
- Fix: `screenshot` method does now support Reflection Probe lighting
- Fix: Lightmapped objects with MultiMaterial objects spamming warnings
- Change: Improve Postprocessing Bloom
- Change: Improve FBX loading
- Change: ContactShadows `autoFit` increases shadow area by factor 1 (previously 0.5)
- Change: Bump three to 0.162.4 adding GLTFExporter plugin hooks and animation retargeting using `userData { name }`
- Change: Bump postprocessing to ^6.35.5
- Change: Bump gltf-progressive dependency

## [3.40.0-alpha.5] - 2024-06-14
- Add: start support for detecting and loading FBX files
- Fix: Camera fitting bounding box calculation where `instanceof Mesh` fails
- Fix: OrbitControls background click only accept primary clicks
- Fix: Issue on Quest due to change in `isMobile` check
- Internal: Deserialization of components can now resolve object if the requested type is an Object3D

## [3.40.0-alpha.2] - 2024-06-12
- Add: Experimental support to load `.fbx` files via `<needle-engine src>`
- Change: Production build command changed - vite plugin now runs when `vite build` is invoked with `--production` - e.g. `vite build -- --production`. The legacy npm build script is automatically updated when the plugin runs for the first time.

## [3.40.0-alpha.1] - 2024-06-10
- Fix: issue where `<needle-engine autoplay>` would not play animations
- Change: "IsMobile" check because window.orientation is deprecated
- Change: USDZ move direct/indirect interaction logic to tapTrigger and add options (by default, both are allowed)

## [3.40.0-alpha] - 2024-06-06
- Update gltf-progressive dependency for smarter LOD texture loading

## [3.39.0-alpha.4] - 2024-06-04
- Add: `WebARSessionRoot.onPlaced` event hook
- Fix: InputField click should open keyboard on Android and iOS
- Fix: ContactShadow and ShadowCatcher meshes should not be raycastable
- Change: OrbitControls `fitCamera` use options object, add "centerCamera" option

## [3.39.0-alpha.3] - 2024-06-04
- Fix: Mesh particle LODs not being loaded
- Fix: UI Image/Raw Image texture LODs not being updated

## [3.39.0-alpha.2] - 2024-06-03
- Fix: vite build pipeline plugin 

## [3.39.0-alpha] - 2024-06-03
- Fix: Issue where OrbitControls setTarget not working as expected sometimes due to first frame matrices not being updated yet
- Fix: Gizmos sometimes rendering for 2 frames instead of just 1
- Change: WebXRImageTracking now applies some smoothing based on jitter amount
- Change: Bump gltf-build-pipeline package to 2.1

## [3.38.0-alpha.3] - 2024-05-30
- Fix: needle menu CSS blur in safari
- Fix: USDZ - move Animation component animations into correct sequence depending on whether it should loop or not
- Fix: QuickLook button being created by WebXR despite `usdzExporter.allowCreateQuicklookButton` explicitly being off

## [3.38.0-alpha.2] - 2024-05-29
- Fix: Issue in vite plugin for Node 16 where fetch wasn't supported yet 
- Fix: AudioListener should remove itself when disabled

## [3.38.0-alpha.1] - 2024-05-29
- Add: OrbitCon...