class · iOS 26.0+ · SDK 27.1
final actor SpeechAnalyzerAnalyzes spoken audio content in various ways and manages the analysis session. The Speech framework provides several modules that can be added to an analyzer to provide specific types of analysis and transcription. Many use cases only need a ``SpeechTranscriber`` module, which performs speech-to-text transcriptions. The `SpeechAnalyzer` class is responsible for: - Holding associated modules - Accepting audio speech input - Controlling the overall analysis Each module is responsible for: - Providing guidance on acceptable input - Providing its analysis or transcription output Analysis is asynchronous. Input, output, and session control are decoupled and typically occur over several different tasks created by you or by the session. In particular, where an Objective-C API might use a delegate to provide results to you, the Swift API's modules provides their results via an `AsyncSequence`. Similarly, you provide speech input to this API via an `AsyncSequence` you create and populate. The analyzer can only analyze one input sequence at a time. ### Perform analysis To perform analysis on audio files and streams, follow these general steps: 1. Create and configure the necessary modules. 2. Ensure the relevant assets are installed or already present. See ``AssetInventory``. 3. Create an input sequence you can use to provide the spoken audio. See helper classes ``AssetInputSequenceProvider`` and ``CaptureInputSequenceProvider``. 4. Create and configure the analyzer with the modules and input sequence. 5. Supply audio. See helper class ``AnalyzerInputConverter``. 6. Start analysis. 7. Act on results. 8. Finish analysis when desired. This example shows how you could perform an analysis that transcribes audio using the `SpeechTranscriber` module: ```swift import Speech // Step 1: Modules guard let locale = SpeechTranscriber.supportedLocale(equivalentTo: Locale.current) else { /* Note unsupported language */ } let transcriber = SpeechTranscriber(locale: locale, preset: .transcription) // Step 2: Assets if let installationRequest = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) { try await installationRequest.downloadAndInstall() } // Step 3: Input sequence let (inputSequence, inputBuilder) = AsyncStream.makeStream(of: AnalyzerInput.self) // Step 4: Analyzer let audioFormat = await SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith: [transcriber]) let analyzer = SpeechAnalyzer(modules: [transcriber]) // Step 5: Supply audio let converter = AnalyzerInputConverter(analyzerFormat: audioFormat) Task { while /* audio remains */ { let buffer = /* Get some audio */ let inputs = try converter.convert(buffer, at: nil) for input in inputs { inputBuilder.yield(input) } } let inputs = try converter.flush() for input in inputs { inputBuilder.yield(input) } inputBuilder.finish() } // Step 7: Act on results Task { do { for try await result in transcriber.results { let bestTranscription = result.text // an AttributedString let plainTextBestTranscription = String(bestTranscription.characters) // a String print(plainTextBestTranscription) } } catch { /* Handle error */ } } // Step 6: Perform analysis let lastSampleTime = try await analyzer.analyzeSequence(inputSequence) // Step 8: Finish analysis if let lastSampleTime { try await analyzer.finalizeAndFinish(through: lastSampleTime) } else { try analyzer.cancelAndFinishNow() } ``` ### Analyze audio from files or capture devices To read audio from a file, asset, or capture device such as a microphone, create an ``AssetInputSequenceProvider`` or ``CaptureInputSequenceProvider`` object. Get the provider object's ``AssetInputSequenceProvider/analyzerInputs`` or ``CaptureInputSequenceProvider/analyzerInputs`` property to convert the source's audio to a supported format and obtain an asynchronous input sequence of the audio. Pass that sequence to ``analyzeSequence(_:)``, ``start(inputSequence:)``, or a similar parameter of the analyzer's initializer. To end the analysis session after processing the audio track or captured audio, call one of the analyzer's `finish` methods. Otherwise, by default, the analyzer won't terminate its result streams and will wait for additional audio input sequences or buffers. See the "Finish analysis" section below for more details. ### Analyze audio from audio buffers You can analyze audio buffers directly without using ``AssetInputSequenceProvider`` or ``CaptureInputSequenceProvider``. To do this: 1. Create an asynchronous input sequence of ``AnalyzerInput`` elements that is appropriate for your use case. 2. Supply the input sequence to ``analyzeSequence(_:)``, ``start(inputSequence:)``, or a similar parameter of the analyzer's initializer. 3. Convert each audio buffer to a supported audio format, either on the fly or in advance. 4. Create ``AnalyzerInput`` objects for each buffer. 5. Add the `AnalyzerInput` objects to the input sequence. To convert `AVAudioBuffer` audio buffers to a supported format as `AnalyzerInput` objects on the fly, use ``AnalyzerInputConverter``. To convert audio buffers to a supported format in advance or with some other technique: 1. Detemine the format to convert to by calling ``bestAvailableAudioFormat(compatibleWith:)-([SpeechModule])`` or individual modules' ``SpeechModule/availableCompatibleAudioFormats`` methods 2. Convert the audio and create `AnalyzerInput` objects as necessary To skip past part of an audio stream, omit the buffers you want to skip from the input sequence. You can resume with a later buffer. When you resume analysis with a later `AVAudioPCMBuffer` buffer, you may need to supply the correct time-code to account for skipped audio. To do this, pass the time-code of the later buffer as the `bufferStartTime` parameter of the corresponding `AnalyzerInput` object. ### Analyze autonomously You can and usually should perform analysis using the ``analyzeSequence(_:)`` or ``analyzeSequence(from:)`` methods; those methods work well with Swift structured concurrency techniques. However, you may prefer that the analyzer proceed independently and perform its analysis autonomously as audio input becomes available in a task managed by the analyzer itself. To use this capability, create the analyzer with one of the initializers that has an input sequence or file parameter, or call ``start(inputSequence:)`` or ``start(inputAudioFile:finishAfterFile:)``. To end the analysis of that input only and start analysis of different input, call one of the `start` methods again. To end the analysis session when the input ends, call ``finalizeAndFinishThroughEndOfInput()``. ### Control processing and timing of results Modules deliver results periodically, but you can manually synchronize their processing and delivery to outside cues. To deliver a result for a particular time-code, call ``finalize(through:)``. To cancel processing of results that are no longer of interest, call ``cancelAnalysis(before:)``. ### Improve responsiveness By default, the analyzer and modules load the system resources that they require lazily, and unload those resources when they're deallocated. To proactively load system resources and "preheat" the analyzer, call ``prepareToAnalyze(in:)`` after setting its modules. This may improve how quickly the modules return their first results. To delay or prevent unloading an analyzer's resources — caching them for later use by a different analyzer instance — you can select a ``SpeechAnalyzer/Options/ModelRetention`` option and create the analyzer with an appropriate ``SpeechAnalyzer/Options`` object. To set the priority of analysis work, create the analyzer with a ``SpeechAnalyzer/Options`` object with the desired `priority` value. Specific modules may also offer options that improve responsiveness. ### Finish analysis To end an analysis session, you must use one of the analyzer's `finish` methods or parameters, or deallocate the analyzer. When the analysis session transitions to the _finished_ state: - The analyzer won't consume additional input from the input sequence (but note that it doesn't drain or terminate the sequence) - Most methods won't do anything; in particular, the analyzer won't accept different input sequences or modules - Module result streams terminate and modules won't publish additional results, though the app can continue to iterate over already-published results > Note: While you can terminate the input sequence you created with a method such as `AsyncStream.Continuation.finish()`, terminating the input sequence does _not_ generally finish the analysis session, and you can continue the session with a different input sequence. (See ``finalizeAndFinishThroughEndOfInput()`` for an exception.) ### Respond to errors When the analyzer or its modules' result streams throw an error, the analysis session becomes finished as described above, and the same error (or a `CancellationError`) is thrown from all waiting methods and result streams. When this happens, you may wish to terminate the input sequence, or create a new analyzer to continue working on the remaining (and any additional) input. ### Manage simultaneous analyses The system normally limits simultaneous analyses to a conservative number, considering hardware capabilities of different devices. If you exceed that number, the system throws an ``SFSpeechError/Code/insufficientResources`` error. However, under certain use cases, the hardware may be able to accommodate additional simultaneous analyses; for example, several simultaneous transcription sessions may use the same language and settings, or only receive audio in an interleaved schedule. To support these use cases, you can override the system to ignore the predefined conservative system resource limits. To override the normal limits, create an analyzer with a ``SpeechAnalyzer/Options`` object with its ``SpeechAnalyzer/Options/ignoresResourceLimits`` value set to `true`. The system allows an unlimited number of analyzers configured with this option. However, the hardware requirements of numerous analyzers will eventually exceed the system's actual capacity, and one or more of the analyzers will fail, throwing an unpredictable error. - Warning: When using this option, test your app on a variety of devices under a variety of scenarios to experimentally determine how many analyzers you can reliably create and expect to function. Consider how to recover in the event one or more analyzers fail.
setContext(newContext: AnalysisContext): Promise<void>final func setContext(_ newContext: AnalysisContext) async throwsSets contextual information to improve or inform the analysis. Other analyzer instances may use the same context object. - Parameter newContext: A context object. This object will replace the current object.
setModules(newModules: Array<SDKHandle>): Promise<void>final func setModules(_ newModules: [any SpeechModule]) async throwsAdds or removes modules. Modules can be added or removed to the analyzer mid-stream. A newly-added module will immediately begin analysis on new audio input, but it will not have access to already-analyzed audio. However, you may keep a copy of previously-analyzed audio and provide it to a separate analyzer. Modules cannot be reused from a different analyzer. - Note: There is a limit to the number of active, backing engine instances and models SpeechAnalyzer is allowed to allocate to clients at the same time. An error will be thrown if that limit is reached in order to prevent poor system performance. Several transcriber instances can share the same backing engine instances and models, so long as the transcribers are configured similarly in certain respects. For iOS and visionOS, this roughly translates to two ongoing recognition instances (or two incompatible modules running simultaneously). There is currently no limit on macOS. - Parameter newModules: A list of modules to include in the analyzer. These modules replace the previous modules, but you may preserve previous modules by including them in the list.
static initInputAudioFileModulesOptionsAnalysisContextFinishAfterFileVolatileRangeChangedHandler(inputAudioFile: _AVFAudio_AVAudioFile, modules: Array<SDKHandle>, options?: (SpeechAnalyzer_Options) | null, analysisContext?: AnalysisContext, finishAfterFile?: boolean, volatileRangeChangedHandler?: (((arg0: _CoreMedia_CMTimeRange, arg1: boolean, arg2: boolean) => void)) | null): Promise<SpeechAnalyzer>convenience init(inputAudioFile: AVAudioFile, modules: [any SpeechModule], options: SpeechAnalyzer.Options? = nil, analysisContext: AnalysisContext = .init(), finishAfterFile: Bool = false, volatileRangeChangedHandler: sending ((CMTimeRange, Bool, Bool) -> Void)? = nil) async throwsCreates an analyzer and begins analysis on an audio file. - Parameters: - inputAudioFile: An audio file opened for reading. - modules: An initial list of modules that will analyze the audio. - options: A structure specifying analysis options. - analysisContext: An object containing contextual information to improve or inform the analysis. - finishAfterFile: If `true`, the analysis will automatically finish after the audio file has been fully processed. Equivalent to calling ``finalizeAndFinishThroughEndOfInput()``. - volatileRangeChangedHandler: A closure called to report the analysis' progress. The closure takes the following parameters: - term range: The current volatile range. - term changedStart: If `true`, the volatile range contains an updated start time. This indicates that prior results have been finalized. - term changedEnd: If `true`, the volatile range contains an update end time. This indicates that analysis of that time has started.
analyzeSequence(audioFile: _AVFAudio_AVAudioFile): Promise<(_CoreMedia_CMTime) | null>final func analyzeSequence(from audioFile: AVAudioFile) async throws -> CMTime?Analyzes an input sequence created from an audio file, returning when the file has been read. When this method returns, the file will have been read, but the last of the audio may still be undergoing analysis. To wait for the analysis to complete, call another method such as ``finalize(through:)`` and await its return. - Parameter audioFile: An `AVAudioFile` opened for reading. - Returns: The time-code of the last audio sample of the input, or `nil` if the file was empty. You may use this value for the parameter of ``finalizeAndFinish(through:)`` (or other methods).
get unownedExecutor(): SDKHandlenonisolated final var unownedExecutor: UnownedSerialExecutor { get }Retrieve the executor for this actor as an optimized, unowned reference. This property must always evaluate to the same executor for a given actor instance, and holding on to the actor must keep the executor alive. This property will be implicitly accessed when work needs to be scheduled onto this actor. These accesses may be merged, eliminated, and rearranged with other work, and they may even be introduced when not strictly required. Visible side effects are therefore strongly discouraged within this property. - SeeAlso: ``SerialExecutor`` - SeeAlso: ``TaskExecutor``
prepareToAnalyzeInWithProgressReadyHandler(audioFormat?: (_AVFAudio_AVAudioFormat) | null, progressReadyHandler?: (((arg0: _Foundation_Progress) => void)) | null): Promise<void>final func prepareToAnalyze(in audioFormat: AVAudioFormat?, withProgressReadyHandler progressReadyHandler: sending ((Progress) -> Void)?) async throwsPrepares the analyzer to begin work with minimal startup delay, reporting the progress of that preparation. - Parameters: - audioFormat: An audio format describing the expected input. The analyzer will load assets appropriate for the given format. If `nil` or if the input is not in this format, the analyzer will reconfigure itself when it processes the actual audio. - progressReadyHandler: A closure that this method calls when progress reporting becomes available. The closure takes the following parameter: - term progress: A `Progress` object that reports the progress of the preparation work.
prepareToAnalyzeIn(audioFormat?: (_AVFAudio_AVAudioFormat) | null): Promise<void>final func prepareToAnalyze(in audioFormat: AVAudioFormat?) async throwsPrepares the analyzer to begin work with minimal startup delay. The analyzer normally performs some configuration lazily as the first audio input becomes available. This method performs that work immediately to reduce or eliminate delays in analyzing the first audio input. - Parameter audioFormat: An audio format describing the expected input. The analyzer will load assets appropriate for the given format. If `nil` or if the input is not in this format, the analyzer will reconfigure itself when it processes the actual audio.
finalizeAndFinish(through: _CoreMedia_CMTime): Promise<void>final func finalizeAndFinish(through: CMTime) async throwsFinishes analysis after finalizing results for a given time-code. This method finalizes like ``finalize(through:)`` and finishes analysis like ``finish(after:)``. - Parameter through: A time-code of the last audio sample that you want to analyze. - Throws: Various errors including `CancellationError` if analysis is finished early before the given input time.
cancelAndFinishNow(): Promise<void>final func cancelAndFinishNow() asyncFinishes analysis immediately. This method cancels all pending work and then finishes analysis. It works similarly to calling ``cancelAnalysis(before:)`` and then ``finish(after:)``, but unlike `finish(after:)`, this method is able to finish analysis prior to any input. The post-conditions for this method are identical to `finish(after:)`. You do not need to call this method before releasing this analyzer or its modules.
static bestAvailableAudioFormatCompatibleWithConsidering(modules: Array<SDKHandle>, naturalFormat?: (_AVFAudio_AVAudioFormat) | null): Promise<(_AVFAudio_AVAudioFormat) | null>static func bestAvailableAudioFormat(compatibleWith modules: [any SpeechModule], considering naturalFormat: AVAudioFormat?) async -> AVAudioFormat?Retrieves the best-quality audio format that the specified modules can work with, taking into account the natural format of the audio and assets installed on the device. Use this method to set up an audio pipeline or pre-convert audio to a usable format. In order to keep `CMTime` values sample-accurate, the analyzer does not transparently upsample, downsample, or convert audio input. - Parameter modules: A list of modules that will be analyzing the audio. - Parameter naturalFormat: An audio format that you prefer to work with, or `nil` if you have no preference. - Returns: `nil` if the specified modules require you to install additional assets.
static bestAvailableAudioFormatCompatibleWith(modules: Array<SDKHandle>): Promise<(_AVFAudio_AVAudioFormat) | null>static func bestAvailableAudioFormat(compatibleWith modules: [any SpeechModule]) async -> AVAudioFormat?Retrieves the best-quality audio format that the specified modules can work with, from assets installed on the device. Use this method to set up an audio pipeline or pre-convert audio to a usable format. In order to keep `CMTime` values sample-accurate, the analyzer does not transparently upsample, downsample, or convert audio input. - Parameter modules: A list of modules that will be analyzing the audio. - Returns: `nil` if the specified modules require you to install additional assets.
finalizeAndFinishThroughEndOfInput(): Promise<void>final func finalizeAndFinishThroughEndOfInput() async throwsFinishes analysis after an audio input sequence has been terminated and fully consumed and the modules' results are finalized. This method waits until the input sequence has been terminated and fully consumed, then finalizes like ``finalize(through:)`` and finishes analysis like ``finish(after:)``. If there is no input sequence, this method waits until there is an input sequence and the sequence terminates. If the input sequence is replaced using one of the `start` methods, this method continues waiting for the replacement input sequence to terminate. This method is primarily useful for autonomous analysis started via one of the `start` methods or a similar initializer. - Throws: `CancellationError` if analysis is finished early before the end of input
start(audioFile: _AVFAudio_AVAudioFile, finishAfterFile?: boolean): Promise<void>final func start(inputAudioFile audioFile: AVAudioFile, finishAfterFile: Bool = false) async throwsStarts analysis of an input sequence created from an audio file and returns immediately. This method stops the autonomous analysis of the previous input sequence. Any audio that hasn't been consumed will be ignored but audio that has been consumed will continue to be analyzed. To ensure the previous sequence's input is fully consumed, call ``finalize(through:)`` first. - Parameters: - audioFile: An AVAudioFile opened for reading. - finishAfterFile: If `true`, the analysis will automatically finish after the audio file has been fully processed. Equivalent to calling ``finalizeAndFinishThroughEndOfInput()``.
finish(after: _CoreMedia_CMTime): Promise<void>final func finish(after: CMTime) async throwsFinishes analysis once input for a given time is consumed. In most cases, you can call ``finalizeAndFinish(through:)`` or ``cancelAndFinishNow()`` instead. Those methods also finish analysis. At the return of this method, the modules' result streams will have ended and the modules will not accept further input from the input sequence. The analyzer will not be able to resume analysis with a different input sequence and will not accept module changes; most methods will do nothing. Analysis of input up to and including the given time may or may not have been completed. Modules will not publish _additional_ results to their streams, but the application can read any results the modules have _already_ published. To ensure analysis is completed or skipped before finishing, call ``finalize(through:)`` or ``cancelAnalysis(before:)``. You do not need to call this method before releasing this analyzer or its modules. - Parameter after: An audio time marking the end of the analysis session. - Throws: `CancellationError` if analysis is finished early before the given input time.
get context(): Promise<AnalysisContext>final var context: AnalysisContext { get async }An object containing contextual information.
static initModulesOptions(modules: Array<SDKHandle>, options?: (SpeechAnalyzer_Options) | null): SpeechAnalyzerconvenience init(modules: [any SpeechModule], options: SpeechAnalyzer.Options? = nil)Creates an analyzer. - Parameters: - modules: An initial list of modules to add to the analyzer. The list can be empty; modules can be added or removed later. - options: A structure specifying analysis options.
finalize(through?: (_CoreMedia_CMTime) | null): Promise<void>final func finalize(through: CMTime?) async throwsFinalizes the modules' analyses. At the return of this method, input up to and including the given time will have been analyzed. Modules will have published the finalized results to their stream, but the application may not have consumed them from the results stream yet. ``volatileRange`` will post-date the given time. If the given time has already been finalized (it pre-dates the volatile range), then this method does nothing. - Parameter through: Finalizes up to and including the given time-code. If the analyzer hasn't already consumed that audio from the input sequence, the method waits until the analyzer consumes that audio before finalizing. If `nil`, finalizes up to and including the last audio the analyzer _has_ consumed from the input sequence, and if the analyzer has not consumed any audio from the input sequence, this method does nothing. - Throws: Various errors including `CancellationError` if analysis is finished early before the given input time
7 declarations were excluded from these bindings. Each reason comes from the generator or the iOS SDK compiler.
SpeechAnalyzer.init(inputSequence:modules:options:analysisContext:volatileRangeChangedHandler:)
This declaration needs Swift generic type arguments. (no canonical witness for InputSequence)
SpeechAnalyzer.modules
The iOS SDK compiler rejected this binding: actor-isolated property 'modules' can not be referenced from a nonisolated context
SpeechAnalyzer.start(inputSequence:)
This declaration needs Swift generic type arguments. (no canonical witness for InputSequence)
SpeechAnalyzer.analyzeSequence(_:)
This declaration needs Swift generic type arguments. (no canonical witness for InputSequence)
SpeechAnalyzer.cancelAnalysis(before:)
The iOS SDK compiler rejected this binding: call to actor-isolated instance method 'cancelAnalysis(before:)' in a synchronous nonisolated context [#ActorIsolatedCall]
SpeechAnalyzer.volatileRange
The iOS SDK compiler rejected this binding: actor-isolated property 'volatileRange' can not be referenced from a nonisolated context
SpeechAnalyzer.setVolatileRangeChangedHandler(_:)
The iOS SDK compiler rejected this binding: call to actor-isolated instance method 'setVolatileRangeChangedHandler' in a synchronous nonisolated context [#ActorIsolatedCall]