Re: [edk2-devel] VirtIO Sound Driver (GSoC 2021)

["Ethin Probst" <harlydavidsen@gmail.com>]

> Is there any necessity for audio input and output to be implemented within the same protocol?  Unlike a network device (which is intrinsically bidirectional), it seems natural to conceptually separate audio input from audio output.

Nope, there isn't a necessity to make them in one, they can be
separated into two.

> The code controlling volume/mute may not have any access to the sample buffer.  The most natural implementation would seem to allow for a platform to notice volume up/down keypresses and use those to control the overall system volume, without any knowledge of which samples (if any) are currently being played by other code in the system.

Your assuming that the audio device your implementing the
volume/muting has volume control and muting functionality within
itself, then. This may not be the case, and so we'd need to
effectively simulate it within the driver, which isn't too hard to do.
As an example, the VirtIO driver does not have a request type for
muting or for volume control (this would, most likely, be within the
VIRTIO_SND_R_PCM_SET_PARAMS request, see sec. 5.14.6.4.3). Therefore,
either the driver would have to simulate the request or return
EFI_UNSUPPORTED this instance.

> Consider also the point of view of the developer implementing a driver for some other piece of audio hardware that happens not to support precisely the same sample rates etc as VirtIO.  It would be extremely ugly to force all future hardware to pretend to have the same capabilities as VirtIO just because the API was initially designed with VirtIO in mind.

Precisely, but the brilliance of VirtIO is that the sample rate,
sample format, &c., do not have to all be supported by a VirtIO
device. Notice, also, how in my protocol proposal I noted that the
sample rates, at least, were "recommended," not "required." Should a
device not happen to support a sample rate or sample format, all it
needs to do is return EFI_INVALID_PARAMETER. Section 5.14.6.2.1
(VIRTIO_SND_R_JACK_GET_CONFIG) describes how a jack tells you what
sample rates it supports, channel mappings, &c.

I do understand how just using a predefined sample rate and sample
format might be a good idea, and if that's the best way, then that's
what we'll do. The protocol can always be revised at a later time if
necessary. I apologize if my stance seems obstinate.

Also, thank you, Laszlo, for your advice -- I hadn't considered that a
network driver would be another good way of figuring out how async
works in UEFI.

On 4/15/21, Andrew Fish <afish@apple.com> wrote:
>
>
>> On Apr 15, 2021, at 5:07 AM, Michael Brown <mcb30@ipxe.org> wrote:
>>
>> On 15/04/2021 06:28, Ethin Probst wrote:
>>> - I hoped to add recording in case we in future want to add
>>> accessibility aids like speech recognition (that was one of the todo
>>> tasks on the EDK2 tasks list)
>>
>> Is there any necessity for audio input and output to be implemented within
>> the same protocol?  Unlike a network device (which is intrinsically
>> bidirectional), it seems natural to conceptually separate audio input from
>> audio output.
>>
>>> - Muting and volume control could easily be added by just replacing
>>> the sample buffer with silence and by multiplying all the samples by a
>>> percentage.
>>
>> The code controlling volume/mute may not have any access to the sample
>> buffer.  The most natural implementation would seem to allow for a
>> platform to notice volume up/down keypresses and use those to control the
>> overall system volume, without any knowledge of which samples (if any) are
>> currently being played by other code in the system.
>>
>
> I’ve also thought of adding NVRAM variable that would let setup, UEFI Shell,
> or even the OS set the current volume, and Mute. This how it would be
> consumed concept is why I proposed mute and volume being separate APIs. The
> volume up/down API in addition to fixed percentage might be overkill, but it
> does allow a non liner mapping to the volume up/down keys. You would be
> surprised how picky audiophiles can be and it seems they like to file
> Bugzillas.
>
>>> - Finally, the reason I used enumerations for specifying parameters
>>> like sample rate and stuff was that I was looking at this protocol
>>> from a general UEFI applications point of view. VirtIO supports all of
>>> the sample configurations listed in my gist, and it seems reasonable
>>> to allow the application to control those parameters instead of
>>> forcing a particular parameter configuration onto the developer.
>>
>> Consider also the point of view of the developer implementing a driver for
>> some other piece of audio hardware that happens not to support precisely
>> the same sample rates etc as VirtIO.  It would be extremely ugly to force
>> all future hardware to pretend to have the same capabilities as VirtIO
>> just because the API was initially designed with VirtIO in mind.
>>
>> As a developer on the other side of the API, writing code to play sound
>> files on an arbitrary unknown platform, I would prefer to simply consume
>> as simple as possible an abstraction of an audio output protocol and not
>> have to care about what hardware is actually implementing it.
>>
>
> It may make sense to have an API to load the buffer/stream and other APIs to
> play or pause. This could allow an optional API to configure how the stream
> is played back. If we add a version to the Protocol that would at least
> future proof us.
>
> We did get feedback that it is very common to speed up the auto playback
> rates for accessibility. I’m not sure if that is practical with a simple PCM
> 16 wave file with the firmware audio implementation. I guess that is
> something we could investigate.
>
> In terms of maybe adding text to speech there is an open source project that
> conceptually we could port to EFI. It would likely be a binary that would
> have to live on the EFI System Partition due to size. I was thinking that
> words per minute could be part of that API and it would produce a PCM 16
> wave file that the audio protocol we are discussing could play.
>
>> Both of these argue in favour of defining a very simple API that expresses
>> only a common baseline capability that is plausibly implementable for
>> every piece of audio hardware ever made.
>>
>> Coupled with the relatively minimalistic requirements for boot-time audio,
>> I'd probably suggest supporting only a single format for audio data, with
>> a fixed sample rate (and possibly only mono output).
>>
>
> In my world the folks that work for Jony asked for a stereo boot bong to
> transition from left to right :). This is not the CODEC you are looking for
> was our response…. I also did not mention that some languages are right to
> left, as the only thing worse than one complex thing is two complex things
> to implement.
>
>> As always: perfection is achieved, not when there is nothing more to add,
>> but when there is nothing left to take away.  :)
>>
>
> "Simplicity is the ultimate sophistication”
>
> Thanks,
>
> Andrew Fish
>
>> Thanks,
>>
>> Michael
>>
>>
>> 
>>
>>
>
>


-- 
Signed,
Ethin D. Probst