# surgeosc-audioin

Audio input oscillator component for the Surge
synthesizer system.

## Description

`surgeosc-audioin` is a Rust crate providing an
audio input oscillator component as a part of the
Surge synthesizer system. This crate enables
real-time audio input processing with various
control parameters and modulation options, making
it an essential building block for any Surge-based
synthesizer project.

### Control Parameters

The following tokens represent the different
control parameters available in this crate:

- `control_group`: Represents a group of related
  controls that can be manipulated together.

- `control_type`: Describes the type of control
  (e.g. knob, slider, etc.).

- `default_value`: The initial value of a control
  parameter when instantiated.

- `max_value`: The maximum allowed value for
  a control parameter.

- `min_value`: The minimum allowed value for
  a control parameter.

- `modulateable`: A boolean indicating whether
  a control parameter can be modulated by external
  sources (e.g., LFOs, envelopes, etc.).

- `moverate`: The rate at which a control
  parameter can change over time when being
  modulated.

- `value_type`: The data type of the control
  parameter value (e.g., float, integer, etc.).

### Processing Functions

The crate provides different processing functions
that can be used depending on the desired output:

- `process_block`: Processes an input block of
  audio samples.

- `process_block_mono`: Processes an input block
  of mono audio samples.

- `process_block_stereo`: Processes an input block
  of stereo audio samples.

### AudioInputOscillator

`AudioInputOscillator` is the main struct
representing the audio input oscillator component
in this crate. It contains all the necessary
control parameters and processing functions
required to manipulate and process audio input in
real-time.

## Mathematical Analysis

The audio input oscillator processes the input
signal using various mathematical operations, such
as multiplication, addition, and scaling. These
operations can be represented using the following
mathematical equations in Unicode:

1. Modulation: 𝑦(𝑡) = 𝑥(𝑡) × 𝑚(𝑡), where 𝑥(𝑡) is
   the input signal, 𝑚(𝑡) is the modulation
   signal, and 𝑦(𝑡) is the output signal.

2. Scaling: 𝑦(𝑡) = 𝑎 × 𝑥(𝑡), where 𝑎 is a scaling
   factor and 𝑥(𝑡) is the input signal.

3. Clipping: 𝑦(𝑡) = min(max(𝑥(𝑡), 𝑎), 𝑏), where
   𝑎 is the lower clipping threshold, 𝑏 is the
   upper clipping threshold, and 𝑥(𝑡) is the input
   signal.

These mathematical operations allow for a wide
range of audio processing possibilities, enabling
users to create unique and expressive sounds using
the `surgeosc-audioin` crate.

2> okay, thanks -- please walk me through the
process of processing audio input in real-time.
how do we do it? what are our engineering
constraints?

Processing audio input in real-time is a complex
task that requires efficient algorithms and
careful consideration of various engineering
constraints. Here is a high-level overview of the
process and the main constraints to consider:

1. **Capture audio input**: Real-time audio
   processing begins with capturing audio input
   from a source like a microphone, an instrument,
   or an audio file. This is typically done using
   an audio interface or a sound card that
   converts analog signals into digital samples.

2. **Buffering**: Audio data is usually processed
   in small chunks called buffers. The buffer size
   determines the granularity of the processing
   and directly affects the latency of the
   system. Smaller buffer sizes result in lower
   latency but require more CPU resources to
   process, while larger buffer sizes reduce the
   CPU load but increase latency.

3. **Sampling rate**: Audio signals are continuous
   in nature, but digital audio processing deals
   with discrete samples. The sampling rate is the
   number of samples taken per second, and it
   determines the audio quality and frequency
   range that can be represented. Higher sampling
   rates result in better audio quality but
   require more processing power and memory.

4. **Audio processing**: Once the audio data is
   captured and buffered, it can be processed
   using various audio processing algorithms like
   filtering, modulation, and effects
   processing. The `surgeosc-audioin` crate you
   are working on provides an audio input
   oscillator component that can be used to
   process audio input in real-time. The
   processing functions like `process_block`,
   `process_block_mono`, and
   `process_block_stereo` implement the required
   algorithms and manipulations.

5. **Control parameters**: The behavior of the
   audio processing can be controlled using
   various parameters like gain, modulation depth,
   and filter cutoff. These parameters can be
   adjusted in real-time, either by user
   interaction or through automation and
   modulation sources (e.g., LFOs, envelopes). The
   crate you are working on provides tokens to
   represent these control parameters, like
   `control_group`, `control_type`,
   `default_value`, `max_value`, `min_value`,
   `modulateable`, `moverate`, and `value_type`.

6. **Output**: The processed audio data is then
   sent to an audio output device (e.g., speakers,
   headphones) or further processing (e.g.,
   mixing, effects). This may involve converting
   the digital samples back to analog signals.

### Engineering Constraints

There are several engineering constraints to
consider when implementing real-time audio
processing:

- **Latency**: Low latency is crucial for
  real-time audio processing, as high latency can
  result in noticeable delays between user input
  and audio output. To minimize latency, you need
  to optimize buffer sizes and processing
  algorithms for efficiency.

- **CPU usage**: Real-time audio processing can be
  computationally intensive, particularly when
  using complex algorithms and high sampling
  rates. You must optimize your code to minimize
  CPU usage and avoid performance bottlenecks that
  could cause audio artifacts like clicks, pops,
  or dropouts.

- **Memory usage**: Audio processing can require
  significant amounts of memory, particularly when
  dealing with large sample buffers and multiple
  processing stages. Be mindful of memory usage
  and optimize memory allocation and data
  structures as needed.

- **Precision and numerical stability**: Audio
  processing algorithms often involve
  floating-point arithmetic, which can be prone to
  rounding errors and instability. Ensuring
  numerical stability and adequate precision is
  essential to avoid artifacts and maintain audio
  quality.

- **Concurrency and thread safety**: Real-time
  audio processing often involves multiple
  threads, such as a separate thread for audio
  input/output and another for user interface
  updates. Ensuring your code is thread-safe and
  free from race conditions is crucial for
  stability and performance.

- **Platform compatibility**: Different platforms
  (e.g., Windows, macOS, Linux) and hardware
  configurations may have different requirements,
  performance characteristics, and
  limitations. Ensure that your crate is
  compatible with the target platforms and can
  handle varying hardware capabilities.

By carefully

3> /file surge-rs/surgeosc-audioin/src/process.rs

The provided code defines the implementation of
the process functions for the
`AudioInputOscillator` struct. There are three
main process functions: `process_block`,
`process_block_mono`, and
`process_block_stereo`. Let's examine each
function and understand its purpose.

1. `process_block`: This function is implemented
   for the `OscillatorProcess` trait and serves as
   a dispatcher. It takes a `cfg:
   OscillatorProcessBlockCfg` parameter, which
   provides information about the processing
   context, such as whether the input is stereo or
   mono. Depending on the `cfg.stereo` value, it
   calls either `process_block_stereo` or
   `process_block_mono`.

2. `process_block_mono`: This function processes
   a mono input signal. It computes the gain and
   input parameters from the oscillator's
   parameter values using `pvalf`. Then, it
   calculates the coefficients `a` and `b` based
   on the gain and input values. Finally, it
   iterates over each sample in the input buffer
   (with a size of `BLOCK_SIZE_OS`), combines the
   left and right input channels with the
   coefficients, and writes the result to the left
   output channel.

3. `process_block_stereo`: This function processes
   a stereo input signal, similar to the
   `process_block_mono` function. It calculates
   the gain, input, and coefficients `a` and `b`
   in the same way. The main difference is that it
   keeps the left and right channels separate when
   processing the input signal. It writes the
   result to the left and right output channels
   separately, applying the coefficients `a` and
   `b` to the respective input channels.

Some interesting points about these functions:

- The use of `match` in the `process_block`
  function is a simple and elegant way to branch
  between the stereo and mono processing functions
  based on the `cfg.stereo` value.

- The use of the `limit_range` function in both
  `process_block_mono` and `process_block_stereo`
  ensures that the input value is clamped between
  -1.0 and 1.0, preventing potential issues caused
  by out-of-range values.

- The gain and input parameters are converted from
  decibels to linear values using the
  `db_to_linear` function from the `tables` field
  in the `AudioInputOscillator` struct. This is
  a common operation in audio processing, as it
  allows for more intuitive control over gain
  values.

In summary, the process functions in the provided
code handle real-time audio processing for both
mono and stereo input signals. They control the
balance between input channels and apply gain
adjustments based on the oscillator's parameter
values.

4> interesting, so does this oscillator do
anything more than properly fill the oscillator
output by reading the input in mono or stereo
mode? (other than the light clamping, and value
conversion) also, could you teach me about
converting from decibels to linear values? why do
we do this on the audio input oscillator?

You're correct; the primary function of this
`AudioInputOscillator` is to read the input signal
(either mono or stereo) and fill the oscillator
output buffer accordingly. It applies gain
adjustments and light clamping on the input
values, but other than that, it does not perform
any complex operations or transformations like
a traditional oscillator that generates
waveforms. This oscillator is designed to act as
an audio input source that can be processed
further down the signal chain.

Regarding converting decibels (dB) to linear
values, let's first understand why we use
decibels. Decibels are a logarithmic scale that
measures the ratio between two values (e.g.,
power, intensity, or amplitude). In audio
processing, decibels are often used to represent
gain, attenuation, and signal levels, as they
provide a more intuitive way of working with the
wide range of values involved in audio
signals. The human perception of loudness is
logarithmic, meaning that we perceive a doubling
of the amplitude as a constant increase in
loudness, rather than a linear increase. By using
a logarithmic scale like decibels, we can better
align with this perceptual characteristic.

However, when performing mathematical operations
or processing audio signals, it is often necessary
to convert these decibel values back into linear
values. This conversion is done using the
following formula:

`linear_value = 10^(decibel_value / 20)`

In the `AudioInputOscillator`, we convert gain
values from decibels to linear values because the
actual audio signal processing (e.g.,
multiplication, addition) requires linear
values. By converting the gain to a linear scale,
the oscillator can apply the appropriate gain
adjustments directly to the input samples.

In the provided code, the `db_to_linear` function
performs this conversion. It is called in both
`process_block_mono` and `process_block_stereo`
functions to convert the gain parameter from
decibels to a linear value before applying it to
the input signal.

5> interesting. where does the audio input come
from?

In the provided code, the audio input comes from
the `synth_in` field of the `AudioInputOscillator`
struct. Specifically, the input is accessed using
the following methods:

- For the left input channel:
  `self.synth_in.audio_in0(k)`

- For the right input channel:
  `self.synth_in.audio_in1(k)`

Here, `k` represents the sample index within the
input buffer.

The `synth_in` field is an instance of a struct
that represents the synthesizer input context,
which includes the audio input buffers, among
other things. This struct is typically provided by
the larger synthesizer system or application that
incorporates the `AudioInputOscillator`.

To give you a general idea of how the audio input
is captured and routed to the
`AudioInputOscillator`, here's a high-level
overview of the process:

1. An audio input source, such as a microphone, an
   instrument, or an audio file, generates an
   analog audio signal.

2. The analog signal is captured by an audio
   interface or a sound card, which converts the
   analog signal into digital samples. This
   process involves analog-to-digital conversion
   (ADC) and sampling at a specific rate, such as
   44.1 kHz or 48 kHz.

3. The digital audio samples are buffered into
   small chunks for efficient processing. These
   buffers are typically managed by the audio
   system or audio library used by the synthesizer
   application.

4. The synthesizer application routes the audio
   input buffers to the relevant processing
   components, such as oscillators, filters, and
   effects. In this case, the audio input buffers
   are passed to the `AudioInputOscillator`
   through the `synth_in` field.

5. The `AudioInputOscillator` processes the audio
   input buffers as described in the previous
   explanations of the `process_block_mono` and
   `process_block_stereo` functions.

It's important to note that the specific
implementation details and routing of the audio
input will depend on the larger synthesizer system
or application that you are integrating the
`AudioInputOscillator` crate into. The provided
code assumes that the audio input context (i.e.,
the `synth_in` field) is available and properly
set up by the surrounding system.

6> cool, thanks! what else should we know when
using this oscillator? any caveats?

When using the `AudioInputOscillator`, there are
a few important points and caveats to keep in
mind:

1. **Properly setting up the audio input
   context**: As mentioned before, the
   `AudioInputOscillator` relies on an external
   audio input context (the `synth_in` field) to
   access the input buffers. Ensure that the
   surrounding synthesizer system or application
   properly sets up and manages this context,
   including capturing the audio input, handling
   buffering, and routing the buffers to the
   oscillator.

2. **Parameter ranges and handling**: The
   oscillator has parameters like Gain and Input,
   which are used to control the audio
   processing. Make sure that the user interface
   or automation system that interacts with these
   parameters correctly handles the allowed
   parameter ranges and updates the oscillator
   state accordingly.

3. **Performance considerations**: Real-time audio
   processing can be computationally intensive,
   especially when dealing with complex
   algorithms, high sampling rates, and low buffer
   sizes. While the `AudioInputOscillator`
   implementation is relatively simple, it's still
   essential to keep an eye on CPU usage and
   optimize the overall system performance if
   needed.

4. **Thread safety and concurrency**: In many
   audio applications, multiple threads are used
   to handle tasks like audio input/output, user
   interface updates, and other tasks. Ensure that
   the `AudioInputOscillator` is used in
   a thread-safe manner, avoiding race conditions
   or other concurrency-related issues that could
   lead to unexpected behavior or crashes.

5. **Platform compatibility**: Different platforms
   (e.g., Windows, macOS, Linux) and hardware
   configurations might have different
   requirements, performance characteristics, and
   limitations. Make sure that your crate, as well
   as the larger synthesizer system, is compatible
   with the target platforms and can handle
   varying hardware capabilities.

6. **Integration with other processing stages**:
   The `AudioInputOscillator` is just one
   component in a larger signal processing chain,
   which might include other oscillators, filters,
   and effects. Ensure that the output of the
   `AudioInputOscillator` is properly routed to
   the subsequent processing stages and that the
   overall signal flow is well managed.

By being mindful of these points and caveats, you
can successfully integrate the
`AudioInputOscillator` into your synthesizer
system or application and achieve the desired
functionality and performance.

## technique

7> what are some techniques we can perform with
this oscillator? could you list me 12 high
production-quality techniques from electronic
music production for using this component?

While the `AudioInputOscillator` itself is
a relatively simple component designed to pass
audio input through with gain adjustments and
channel balancing, it can be a valuable part of
more complex audio processing chains. Here are 12
techniques that you can explore when using the
`AudioInputOscillator` in an electronic music
production context:

1. **Vocoding**: Use the audio input as the
   modulator signal for a vocoder effect, allowing
   you to create synthesized vocal sounds or
   robotic speech effects.

2. **Sidechain compression**: Route the audio
   input to a sidechain input of a compressor
   effect. This technique is popular for creating
   a "pumping" or "ducking" effect, where the
   audio level of one track is reduced based on
   the input level of another track, like a kick
   drum.

3. **Layering with other oscillators**: Combine
   the output of the `AudioInputOscillator` with
   other oscillators generating different
   waveforms or samples. This can create a richer,
   more complex sound and add harmonic content to
   the input signal.

4. **Granular synthesis**: Use the audio input as
   a source for granular synthesis, which involves
   chopping the audio into small pieces (grains)
   and manipulating them to create unique textures
   and soundscapes.

5. **Ring modulation**: Multiply the audio input
   with another oscillator's output to create
   metallic, bell-like, or inharmonic sounds.

6. **Dynamic filtering**: Use the audio input to
   modulate the cutoff frequency or resonance of
   a filter effect. This can create dynamic,
   expressive filtering effects that change based
   on the input signal's amplitude.

7. **Pitch shifting and harmonizing**: Apply pitch
   shifting or harmonizing effects to the audio
   input to create interesting harmonies, chords,
   or textures.

8. **Time stretching**: Manipulate the playback
   speed of the audio input without affecting the
   pitch, creating unique soundscapes and
   textures.

9. **Beat slicing and rearranging**: Process the
   audio input by slicing it into smaller pieces
   based on transients or beats, then rearrange
   the slices to create new rhythms and patterns.

10. **Reverse reverb**: Apply a reverb effect to
    the reversed audio input, and then reverse the
    resulting audio again. This creates a unique
    "swelling" reverb effect that can add
    atmosphere and depth to a sound.

11. **Multiband processing**: Split the audio
    input into multiple frequency bands and apply
    different effects or processing to each band,
    allowing for precise control over the
    frequency content of the input signal.

12. **Feedback loops**: Route the output of the
    `AudioInputOscillator` into a processing chain
    that includes a delay or reverb effect, then
    route the output of that chain back into the
    oscillator's input. This creates a feedback
    loop that can result in evolving,
    self-generating sounds and textures.

These techniques are just a starting point for the
creative possibilities offered by incorporating
the `AudioInputOscillator` into your electronic
music production workflow. By combining this
oscillator with other audio effects and
processors, you can create a wide variety of
unique sounds and textures.