RealityFoundation · IOS API REFERENCE

VideoMaterial

struct · iOS 14.0+ · SDK 27.1

struct VideoMaterial

A material that supports animated textures. In RealityKit, a _material_ is an object that defines the surface properties of a rendered 3D object. A `VideoMaterial` is a material that maps a movie file on to the surface of an entity. Video materials are _unlit_, which means that scene lighting doesn’t affect them. Video materials support transparency if the source video’s file format also supports transparency. Video materials use an <doc://com.apple.documentation/documentation/avfoundation/avplayer> instance to control movie playback. You can use any movie file format that <doc://com.apple.documentation/documentation/avfoundation/avplayer> supports to create a video material. To control playback of the material’s video, use the ``VideoMaterial/avPlayer`` property, which offers methods like <doc://com.apple.documentation/documentation/avfoundation/avplayer/1386726-play> and <doc://com.apple.documentation/documentation/avfoundation/avplayer/1387895-pause>. The following code demonstrates how to create and start playing a video material using a movie file from your application bundle. ```swift // Create a URL that points to the movie file. if let url = Bundle.main.url(forResource: "MyMovie", withExtension: "mp4") { // Create an AVPlayer instance to control playback of that movie. let player = AVPlayer(url: url) // Instantiate and configure the video material. let material = VideoMaterial(avPlayer: player) // Configure audio playback mode. material.controller.audioInputMode = .spatial // Create a new model entity using the video material. let modelEntity = ModelEntity(mesh: cube, materials: [material]) // Start playing the video. player.play() } ``` To see an example of using a video texture in RealityKit, see <doc://com.apple.documentation/documentation/realitykit/creating_a_game_with_scene_understanding>.

Members (14)

get · iOS 14.0+

controller

TypeScript

get controller(): VideoPlaybackController

Swift declaration

var controller: VideoPlaybackController { get }

An object that configures framework-specific video options. Use this property to configure AR-specific properties of the texture’s video, such as whether the material should use spatial audio. The following example demonstrates enabling spatial audio for a video material: ```swift material.controller.audioInputMode = .spatial ```

get · iOS 18.0+

readsDepth

TypeScript

get readsDepth(): boolean

Swift declaration

var readsDepth: Bool { get set }

A boolean value that determines whether this material performs the depth test by reading RealityKit's depth buffer. If true, meshes with this material will depth test each of their fragments when being rendered. If an object that writes depth is in front of this material, this material will be hidden. If false, meshes with this material will ignore the depth test, and always render all of their fragments during their draw call, regardless of the positioning of other objects in the scene. Note that other objects may still render on top of this material, depending on draw order. The default value is true.

set · iOS 18.0+

readsDepth

TypeScript

set readsDepth(value: boolean)

Swift declaration

var readsDepth: Bool { get set }

get · iOS 18.0+

faceCulling

TypeScript

get faceCulling(): MaterialParameterTypes_FaceCulling

Swift declaration

var faceCulling: VideoMaterial.FaceCulling { get set }

A process in which the system specifies polygons to remove before rendering a mesh using this material. To improve performance, RealityKit culls polygons, or faces, that it determines won’t be visible. Discarding faces that aren’t part of the final render eliminates the need to do any calculations for those faces. RealityKit recognizes when a face aims toward the camera (a front face) or away from the camera (a back face). This value controls the type of faces RealityKit culls. This value defaults to ``MaterialParameterTypes/FaceCulling/back``, which means RealityKit removes faces that point away from the camera. Most of the time, this is the correct behavior, because back faces are usually obscured by other front-facing polygons. You can change the culling behavior to cull front faces instead or to turn off face culling altogether, but be aware that turning off face culling results in less efficient rendering and may negatively impact your app’s frame rate.

set · iOS 18.0+

faceCulling

TypeScript

set faceCulling(value: MaterialParameterTypes_FaceCulling)

Swift declaration

var faceCulling: VideoMaterial.FaceCulling { get set }

get · iOS 18.0+

writesDepth

TypeScript

get writesDepth(): boolean

Swift declaration

var writesDepth: Bool { get set }

A boolean value that determines whether this material writes its depth into RealityKit's depth buffer. If true, meshes with this material will occlude objects behind them by writing their depth into RealityKit's depth buffer. If false, meshes with this material will not write their depth into RealityKit's depth buffer, and may be overwritten by objects drawn behind them, depending on draw order. The default value is true.

set · iOS 18.0+

writesDepth

TypeScript

set writesDepth(value: boolean)

Swift declaration

var writesDepth: Bool { get set }

init · iOS 18.0+

init

TypeScript

static initVideoRenderer(videoRenderer: _AVFoundation_AVSampleBufferVideoRenderer): VideoMaterial

Swift declaration

init(videoRenderer: AVSampleBufferVideoRenderer)

Creates and initializes a video material for a sample buffer video renderer object. To create a `VideoMaterial`, first create an <doc://com.apple.documentation/documentation/avfoundation/avsamplebuffervideorenderer> instance with no parameters and then pass that to the `VideoMaterial` initializer. After creating the `VideoMaterial`, create a <doc://com.apple.documentation/documentation/avfoundation/avassetreader> with the <doc://com.apple.documentation/documentation/avfoundation/avurlasset> and load the media (video), which you need to track as the <doc://com.apple.documentation/documentation/avfoundation/avassettrack>. Feed this `AVAssetTrack` to the <doc://com.apple.documentation/documentation/avfoundation/avassetreadertrackoutput> and add this output tracker to the `AVAssetReader` you created before. Wait for setup to finish and start reading. Read the sample buffers from the reader and enqueue it into the `AVSampleBufferVideoRenderer` object. `AVSampleBufferVideoRenderer` uses a push model, so feed the video sample buffers until the queue is full and then resume feeding it when it's ready for more. You can use `isReadyForMoreMediaData` on the `AVSampleBufferVideoRenderer` object in a while-loop to check if you need to enqueue more sample buffers. You can't use the same `AVSampleBufferVideoRenderer` object with more than one `VideoMaterial`. You need to synchronize the audio, captions, and playback rate separately in your application. The following code example demonstrates how to synchronize video with audio: ```swift // Create an `AVSampleBufferVideoRenderer` instance to control playback of a movie. let videoRenderer = AVSampleBufferVideoRenderer() // Create an `AVSampleBufferAudioRenderer` instance to control audio of the playback. let audioRenderer = AVSampleBufferAudioRenderer() // Create a `AVSampleBufferRenderSynchronizer` instance to synchronize video and audio. let synchronizer = AVSampleBufferRenderSynchronizer() // Add both videoRenderer and audioRenderer to the synchronizer. synchronizer.addRenderer(videoRenderer) synchronizer.addRenderer(audioRenderer) // Create an entity for display. let entity = ModelEntity() // Create a `VideoMaterial` object that supplies the `AVSampleBufferVideoRenderer` object. entity.model = .init(mesh: plane, materials: [VideoMaterial(videoRenderer: videoRenderer)]) // Create a URL that points to the movie file. if let url = Bundle.main.url(forResource: "MyMovie", withExtension: "mp4") { let sourceAsset = AVURLAsset(url: url) let sourceAssetReader = AVAssetReader(asset: sourceAsset) let sourceAssetVideoTrack = sourceAsset.loadTracks(withMediaType: .video).first let sourceAssetAudioTrack = sourceAsset.loadTracks(withMediaType: .audio).first let sourceAssetReaderVideoTrackOutput = AVAssetReaderTrackOutput(track: sourceAssetVideoTrack!, outputSettings: nil) let sourceAssetReaderAudioTrackOutput = AVAssetReaderTrackOutput(track: sourceAssetAudioTrack!, outputSettings: nil) sourceAssetReader.add(sourceAssetReaderVideoTrackOutput!) sourceAssetReader.add(sourceAssetReaderAudioTrackOutput!) sourceAssetReader.startReading() videoRenderer.requestMediaDataWhenReady(on: DispatchQueue.global()) { while videoRenderer.isReadyForMoreMediaData { if let sampleBuffer = sourceAssetReaderVideoTrackOutput!.copyNextSampleBuffer() { videoRenderer.enqueue(sampleBuffer) } else { videoRenderer.stopRequestingMediaData() return } } } audioRenderer.requestMediaDataWhenReady(on: DispatchQueue.global()) { while audioRenderer.isReadyForMoreMediaData { if let sampleBuffer = sourceAssetReaderAudioTrackOutput!.copyNextSampleBuffer() { audioRenderer.enqueue(sampleBuffer) } else { audioRenderer.stopRequestingMediaData() return } } } // Start the playback immediately. synchronizer.setRate(1, time: .zero) } ``` - Parameters: - videoRenderer: The sample buffer video renderer whose visual contents the material presents.

get · iOS 18.0+

videoRenderer

TypeScript

get videoRenderer(): (_AVFoundation_AVSampleBufferVideoRenderer) | null

Swift declaration

var videoRenderer: AVSampleBufferVideoRenderer? { get }

The material’s video renderer. Pass this renderer to the material as a parameter in the initializer; you can't replace it afterward. You can't use the same `AVSampleBufferVideoRenderer` object with more than one `VideoMaterial`.

get · iOS 18.0+

triangleFillMode

TypeScript

get triangleFillMode(): MaterialParameterTypes_TriangleFillMode

Swift declaration

var triangleFillMode: VideoMaterial.TriangleFillMode { get set }

The object that controls how RealityKit draws triangles. A value of ``MaterialParameterTypes/TriangleFillMode/fill`` causes RealityKit to draw triangles normally, while a value of ``MaterialParameterTypes/TriangleFillMode/lines`` turns on wireframe rendering.

set · iOS 18.0+

triangleFillMode

TypeScript

set triangleFillMode(value: MaterialParameterTypes_TriangleFillMode)

Swift declaration

var triangleFillMode: VideoMaterial.TriangleFillMode { get set }

init · iOS 14.0+

init

TypeScript

static initAvPlayer(avPlayer: _AVFoundation_AVPlayer): Promise<VideoMaterial>

Swift declaration

@MainActor @preconcurrency init(avPlayer: AVPlayer)

Creates a new video material. To create a video material, first create an <doc://com.apple.documentation/documentation/avfoundation/avplayer> instance initialized with a URL that points to the movie file you want the video material to play, then pass that to this initializer. The following code demonstrates this process. ```swift // Create a URL that points to the movie file. if let url = Bundle.main.url(forResource: "MyMovie", withExtension: "mp4") { // Create an AVPlayer instance to control playback of that movie. let player = AVPlayer(url: url) // Instantiate and configure the video material. let material = VideoMaterial(avPlayer: player) // Configure audio playback mode. This is optional for movie // files that contain sound. material.controller.audioInputMode = .spatial // Create a new model entity using the video material. let modelEntity = ModelEntity(mesh: cube, materials: [material]) // Start playing the video. player.play() } ``` - Parameters: - avPlayer: An <doc://com.apple.documentation/documentation/avfoundation/avplayer> instance.

get · iOS 14.0+

avPlayer

TypeScript

get avPlayer(): Promise<(_AVFoundation_AVPlayer) | null>

Swift declaration

@MainActor @preconcurrency var avPlayer: AVPlayer? { get set }

The material’s video playback controller. Use this property to control animation playback for a video texture. This property allows you to play or pause the movie, _seek to_ (in other words, skip to) a specific part of the movie, or to change the movie’s playback rate. The following code demonstrates pausing the texture’s video and restarting it from the beginning of the movie file: ```swift myMaterial.avPlayer.pause() myMaterial.avPlayer.seek(to: .zero) myMaterial.avPlayer.play() ```

set · iOS 14.0+

avPlayer

TypeScript

setAvPlayer(value?: (_AVFoundation_AVPlayer) | null): Promise<void>

Swift declaration

@MainActor @preconcurrency var avPlayer: AVPlayer? { get set }

Skipped declarations

No declarations were skipped in this snapshot.

All RealityFoundation types →