Continuation of Audio Output Protocol/UEFI accessibility project from 2021 GSoC

Continuation of Audio Output Protocol/UEFI accessibility project from 2021 GSoC

[Ethin Probst]

Hi all,
I'm writing because I'm considering applying once again for GSoC to 
continue my work on the audio output subsystem, specifically focusing on 
either HDA or USB audio. Last year I came incredibly close to getting 
audio output working, but Qemu did some weird things that I was unaware 
of with the HDA specification so it didn't work. Specifically, it took 
liberties with the interpretation of the RIRB/CORB size values, 
considering the value `0` as `0` entries instead of 256 as mandated by 
the HDA specification. As others on the OSDev forum have noted when they 
discovered this problem, Qemu does just enough to get Linux booting and 
no more. This raises many questions; in particular, what other liberties 
does Qemu take with other specifications that it shouldn't?
However, my Mentor (Leif) and I also came incredibly close to getting 
USB audio working. What prevented us from achieving this goal was the 
fact that USB isochronous transfers were not supported by the UEFI USB 
stack. Thus, we had to switch methodologies and priorities right in the 
middle of the program, which only caused problems for us (though 
primarily me).
After a lot of deliberation on my part in ways I could've done 
significantly better than I did last year, I have come to the conclusion 
that I am jumping way ahead of myself in trying to achieve audio now 
instead of splitting it up into smaller steps and implementing the 
underlying infrastructure. I therefore propose to take a significant 
step backwards and to shelve the audio output itself and to focus, 
instead, on the underlying stacks in question: for example, if we 
continue with the USB audio route -- which may be, as capitalists might 
say, the most profitable route -- it would be in our best interest to 
solely focus on the USB stack and getting the various other USB transfer 
types fully supported before continuing with USB audio. As it currently 
stands (at least last time I checked, anyway, which was a while ago), 
the USB stack is almost entirely focused on HID and UAS support to the 
exclusion of all else, which forbids things like USB audio support. It 
allows things like making braille displays function (since a braille 
display is just another HID device), but I do not have access to a 
physical braille display to test, otherwise I would try my hand at that. 
I believe I could get my hands on one, though, but I'm not sure. 
Ultimately I have three choices and I'd like your advice:
1. Try to continue going for the Intel HDA spec, even though Qemu takes 
at least one liberty with the HDA specification and may take more, which 
will require a lot of digging in the qemu source code;
2. Implement the remaining (missing) USB transfer types into the UEFI 
USB drivers; or
3. Try to get my hands on a braille display and see about implementing 
the USB braille HID standard.
I should also note that I am graduating this May, so my life will be in 
extreme flux and I may need to withdraw from GSoC should I attain a job 
(since I'm job-hunting at the moment). But if that happens, I will let 
you guys know, but I of course would like to continue working on the 
project in my spare time, even if I am not in GSoC.
I look forward to once again working with you guys!

Re: [edk2-devel] Continuation of Audio Output Protocol/UEFI accessibility project from 2021 GSoC

[Nate DeSimone]

[-- Attachment #1: Type: text/plain, Size: 5133 bytes --]

Hey Ethin,

Welcome back! Finishing what one started is certainly a noble endeavor! It is good that you have the root cause of what went wrong last year fully understood and therefore have a good idea of the adjustments you would need to make to be successful this year.

So, the one thing that I like about the USB audio route is that not only will you be making audio work; you will also be improving the core USB driver stack. This will make new unrelated USB drivers that have not been considered yet possible and therefore helps improve the overall state of EDK II in general. The downside of the USB approach is that it will be substantially more work to get done.

With regards to your findings on qemu’s HDA implementation that does not surprise me at all. I have plenty of first-hand experience with devices that do not behave per the spec that they supposedly follow. I’m sure it is possible to build an HDA driver that works around these types of weird behaviors since the OS guys have managed to do it somehow. I think you would want to read the BSD kernel code as they probably already have the workaround figured out.

I suspect the decision on whether to go the USB route or to go the HDA route really comes down how much time you think you will have this summer. The HDA route sounds like a medium project to me, whereas the USB route sounds like a large one. Think about how much you want to sign up for and then go ahead and write up a proposal accordingly 😊. Let me know if you have any questions!

Best Regards,
Nate

From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Ethin Probst <harlydavidsen@gmail.com>
Date: Wednesday, April 13, 2022 at 4:46 PM
To: devel@edk2.groups.io <devel@edk2.groups.io>
Subject: [edk2-devel] Continuation of Audio Output Protocol/UEFI accessibility project from 2021 GSoC
Hi all,
I'm writing because I'm considering applying once again for GSoC to
continue my work on the audio output subsystem, specifically focusing on
either HDA or USB audio. Last year I came incredibly close to getting
audio output working, but Qemu did some weird things that I was unaware
of with the HDA specification so it didn't work. Specifically, it took
liberties with the interpretation of the RIRB/CORB size values,
considering the value `0` as `0` entries instead of 256 as mandated by
the HDA specification. As others on the OSDev forum have noted when they
discovered this problem, Qemu does just enough to get Linux booting and
no more. This raises many questions; in particular, what other liberties
does Qemu take with other specifications that it shouldn't?
However, my Mentor (Leif) and I also came incredibly close to getting
USB audio working. What prevented us from achieving this goal was the
fact that USB isochronous transfers were not supported by the UEFI USB
stack. Thus, we had to switch methodologies and priorities right in the
middle of the program, which only caused problems for us (though
primarily me).
After a lot of deliberation on my part in ways I could've done
significantly better than I did last year, I have come to the conclusion
that I am jumping way ahead of myself in trying to achieve audio now
instead of splitting it up into smaller steps and implementing the
underlying infrastructure. I therefore propose to take a significant
step backwards and to shelve the audio output itself and to focus,
instead, on the underlying stacks in question: for example, if we
continue with the USB audio route -- which may be, as capitalists might
say, the most profitable route -- it would be in our best interest to
solely focus on the USB stack and getting the various other USB transfer
types fully supported before continuing with USB audio. As it currently
stands (at least last time I checked, anyway, which was a while ago),
the USB stack is almost entirely focused on HID and UAS support to the
exclusion of all else, which forbids things like USB audio support. It
allows things like making braille displays function (since a braille
display is just another HID device), but I do not have access to a
physical braille display to test, otherwise I would try my hand at that.
I believe I could get my hands on one, though, but I'm not sure.
Ultimately I have three choices and I'd like your advice:
1. Try to continue going for the Intel HDA spec, even though Qemu takes
at least one liberty with the HDA specification and may take more, which
will require a lot of digging in the qemu source code;
2. Implement the remaining (missing) USB transfer types into the UEFI
USB drivers; or
3. Try to get my hands on a braille display and see about implementing
the USB braille HID standard.
I should also note that I am graduating this May, so my life will be in
extreme flux and I may need to withdraw from GSoC should I attain a job
(since I'm job-hunting at the moment). But if that happens, I will let
you guys know, but I of course would like to continue working on the
project in my spare time, even if I am not in GSoC.
I look forward to once again working with you guys!





[-- Attachment #2: Type: text/html, Size: 8178 bytes --]