From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp02.apple.com (ma1-aaemail-dr-lapp02.apple.com [17.171.2.68]) by mx.groups.io with SMTP id smtpd.web11.12121.1618592638895950562 for ; Fri, 16 Apr 2021 10:03:59 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=D7IMMCWZ; spf=pass (domain: apple.com, ip: 17.171.2.68, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp02.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp02.apple.com (8.16.0.42/8.16.0.42) with SMTP id 13GGxiHD047716; Fri, 16 Apr 2021 10:03:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=20180706; bh=vFYFvVXIw/+hdVuDR4ebWye7h3vClXNUpqiiTGgpe3M=; b=D7IMMCWZD6fLRNomx73iF//Gg7IU415eMG9MYTFIJ+DOLkXZpovYKZdpRIamfaZCPrpA Ai61/90PGl3xh/XH9wFNsG7KV7iliNbUu9tcfYIvA9t+2vWg4w7xy4eKd7233EDh63nd ylbC6kct51OpzUeYA4GO+MYeKB4bmt+WD4Tps/oI6o0ktOUhKyjmkULWTU0O0tj7Oktf vYDaq7r/vMW9EBzLh10t9J/hbd0gAhanVxQAQ7U+sYQipGgyJ7mpJCCt/+oWRhdc8cbY baQG91L3R9KWXjFzyobRvmOkGHBtqH0zMlJ4LKeGWoD9pt9P3XMPioHFZFutHavOeGya QA== Received: from rn-mailsvcp-mta-lapp02.rno.apple.com (rn-mailsvcp-mta-lapp02.rno.apple.com [10.225.203.150]) by ma1-aaemail-dr-lapp02.apple.com with ESMTP id 37ye2n8cwx-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 16 Apr 2021 10:03:58 -0700 Received: from rn-mailsvcp-mmp-lapp03.rno.apple.com (rn-mailsvcp-mmp-lapp03.rno.apple.com [17.179.253.16]) by rn-mailsvcp-mta-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) with ESMTPS id <0QRO0099Y22HA530@rn-mailsvcp-mta-lapp02.rno.apple.com>; Fri, 16 Apr 2021 10:03:53 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp03.rno.apple.com by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) id <0QRO009001WPKU00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Fri, 16 Apr 2021 10:03:53 -0700 (PDT) X-Va-A: X-Va-T-CD: 004664a286f28cb677a60481ed4243ce X-Va-E-CD: 4730c80ee67030d4f2c83e40b4ab0357 X-Va-R-CD: 6f0325faf294bd23a6d751c620be9d51 X-Va-CD: 0 X-Va-ID: a5ee451f-92e3-43ae-b786-d8f54d856272 X-V-A: X-V-T-CD: 004664a286f28cb677a60481ed4243ce X-V-E-CD: 4730c80ee67030d4f2c83e40b4ab0357 X-V-R-CD: 6f0325faf294bd23a6d751c620be9d51 X-V-CD: 0 X-V-ID: 0c68195a-7f35-42a6-921c-5bd8b81719b0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761 definitions=2021-04-16_08:2021-04-16,2021-04-16 signatures=0 Received: from [17.235.19.21] (unknown [17.235.19.21]) by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) with ESMTPSA id <0QRO00PHQ22F9X00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Fri, 16 Apr 2021 10:03:53 -0700 (PDT) MIME-version: 1.0 (Mac OS X Mail 14.0 \(3654.20.0.2.1\)) Subject: Re: [edk2-devel] VirtIO Sound Driver (GSoC 2021) From: "Andrew Fish" In-reply-to: <20210416113447.GG1664@vanye> Date: Fri, 16 Apr 2021 10:03:51 -0700 Cc: Ethin Probst , Michael Brown , Mike Kinney , Laszlo Ersek , "Desimone, Nathaniel L" , Rafael Rodrigues Machado , Gerd Hoffmann Message-id: <10E3436C-D743-4B2F-8E4B-7AD93B82FC92@apple.com> References: <4AEC1784-99AF-47EF-B7DD-77F91EA3D7E9@apple.com> <309cc5ca-2ecd-79dd-b183-eec0572ea982@ipxe.org> <33e37977-2d27-36a0-89a6-36e513d06b2f@ipxe.org> <6F69BEA6-5B7A-42E5-B6DA-D819ECC85EE5@apple.com> <20210416113447.GG1664@vanye> To: edk2-devel-groups-io , Leif Lindholm X-Mailer: Apple Mail (2.3654.20.0.2.1) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761 definitions=2021-04-16_08:2021-04-16,2021-04-16 signatures=0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: quoted-printable > On Apr 16, 2021, at 4:34 AM, Leif Lindholm wrote: >=20 > Hi Ethin, >=20 > I think we also want to have a SetMode function, even if we don't get > around to implement proper support for it as part of GSoC (although I > expect at least for virtio, that should be pretty straightforward). >=20 Leif, I=E2=80=99m think if we have an API to load the buffer and a 2nd API to pl= ay the buffer an optional 3rd API could configure the streams.=20 > It's quite likely that speech for UI would be stored as 8kHz (or > 20kHz) in some systems, whereas the example for playing a tune in GRUB > would more likely be a 44.1 kHz mp3/wav/ogg/flac. >=20 > For the GSoC project, I think it would be quite reasonable to > pre-generate pure PCM streams for testing rather than decoding > anything on the fly. >=20 > Porting/writing decoders is really a separate task from enabling the > output. I would much rather see USB *and* HDA support able to play pcm > streams before worrying about decoding. >=20 I agree it might turn out it is easier to have the text to speech code jus= t encode a PCM directly.=20 Thanks, Andrew Fish > / > Leif >=20 > On Fri, Apr 16, 2021 at 00:33:06 -0500, Ethin Probst wrote: >> Thanks for that explanation (I missed Mike's message). Earlier I sent >> a summary of those things that we can agree on: mainly, that we have >> mute, volume control, a load buffer, (maybe) an unload buffer, and a >> start/stop stream function. Now that I fully understand the >> ramifications of this I don't mind settling for a specific format and >> sample rate, and signed 16-bit PCM audio is, I think, the most widely >> used one out there, besides 64-bit floating point samples, which I've >> only seen used in DAWs, and that's something we don't need. >> Are you sure you want the firmware itself to handle the decoding of >> WAV audio? I can make a library class for that, but I'll definitely >> need help with the security aspect. >>=20 >> On 4/16/21, Andrew Fish via groups.io wro= te: >>>=20 >>>=20 >>>> On Apr 15, 2021, at 5:59 PM, Michael Brown wrote: >>>>=20 >>>> On 16/04/2021 00:42, Ethin Probst wrote: >>>>> Forcing a particular channel mapping, sample rate and sample format = on >>>>> everyone would complicate application code. From an application poin= t >>>>> of view, one would, with that type of protocol, need to do the >>>>> following: >>>>> 1) Load an audio file in any audio file format from any storage >>>>> mechanism. >>>>> 2) Decode the audio file format to extract the samples and audio >>>>> metadata. >>>>> 3) Resample the (now decoded) audio samples and convert (quantize) t= he >>>>> audio samples into signed 16-bit PCM audio. >>>>> 4) forward the samples onto the EFI audio protocol. >>>>=20 >>>> You have made an incorrect assumption that there exists a requirement= to >>>> be able to play audio files in arbitrary formats. This requirement d= oes >>>> not exist. >>>>=20 >>>> With a protocol-mandated fixed baseline set of audio parameters (samp= le >>>> rate etc), what would happen in practice is that the audio files woul= d be >>>> encoded in that format at *build* time, using tools entirely external= to >>>> UEFI. The application code is then trivially simple: it just does "l= oad >>>> blob, pass blob to audio protocol". >>>>=20 >>>=20 >>>=20 >>> Ethin, >>>=20 >>> Given the goal is an industry standard we value interoperability more = that >>> flexibility. >>>=20 >>> How about another use case. Lets say the Linux OS loader (Grub) wants = to >>> have an accessible UI so it decides to sore sound files on the EFI Sys= tem >>> Partition and use our new fancy UEFI Audio Protocol to add audio to th= e OS >>> loader GUI. So that version of Grub needs to work on 1,000 of differen= t PCs >>> and a wide range of UEFI Audio driver implementations. It is a much ea= sier >>> world if Wave PCM 16 bit just works every place. You could add a lot o= f >>> complexity and try to encode the audio on the fly, maybe even in Linux >>> proper but that falls down if you are booting from read only media lik= e a >>> DVD or backup tape (yes people still do that in server land). >>>=20 >>> The other problem with flexibility is you just made the test matrix ve= ry >>> large for every driver that needs to get implemented. For something as >>> complex as Intel HDA how you hook up the hardware and what CODECs you = use >>> may impact the quality of the playback for a given board. Your EFI is = likely >>> going to pick a single encoding at that will get tested all the time i= f your >>> system has audio, but all 50 other things you support not so much. So = that >>> will required testing, and some one with audiophile ears (or an AI pro= gram) >>> to test all the combinations. I=E2=80=99m not kidding I get BZs on the= quality of >>> the boot bong on our systems. >>>=20 >>>=20 >>>>> typedef struct EFI_SIMPLE_AUDIO_PROTOCOL { >>>>> EFI_SIMPLE_AUDIO_PROTOCOL_RESET Reset; >>>>> EFI_SIMPLE_AUDIO_PROTOCOL_START Start; >>>>> EFI_SIMPLE_AUDIO_PROTOCOL_STOP Stop; >>>>> } EFI_SIMPLE_AUDIO_PROTOCOL; >>>>=20 >>>> This is now starting to look like something that belongs in boot-time >>>> firmware. :) >>>>=20 >>>=20 >>> I think that got a little too simple I=E2=80=99d go back and look at t= he example I >>> posted to the thread but add an API to load the buffer, and then play = the >>> buffer (that way we can an API in the future to twiddle knobs). That A= PI >>> also implements the async EFI interface. Trust me the 1st thing that i= s >>> going to happen when we add audio is some one is going to complain in = xyz >>> state we should mute audio, or we should honer audio volume and mute >>> settings from setup, or from values set in the OS. Or some one is goin= g to >>> want the volume keys on the keyboard to work in EFI. >>>=20 >>> Also if you need to pick apart the Wave PCM 16 byte file to feed it in= to the >>> audio hardware that probably means we should have a library that does = that >>> work, so other Audio drivers can share that code. Also having a librar= y >>> makes it easier to write a unit test. We need to be security conscious= as we >>> need to treat the Audo file as attacker controlled data. >>>=20 >>> Thanks, >>>=20 >>> Andrew Fish >>>=20 >>>> Michael >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>=20 >>=20 >> --=20 >> Signed, >> Ethin D. Probst >=20 >=20 >=20 >=20 >=20