GRUB issue on device priority

GRUB issue on device priority

[Haojian Zhuang]

Hi all,

It seems there's a device priority issue in GRUB.

All block io handles are linked into the list in edk2, and GRUB could 
fetch it. Then GRUB creates its own ascending on HD priority.
add_device():
   for (p = devices; *p; p = &((*p)->next)) {
     ret = grub_efi_compare_device_paths (grub_efi_find_last_device_path 
((*p->device_path),
                          grub_efi_find_last_device_path->device_path));
     if (ret == 0) {
       ret = grub_efi_compare_device_paths ((*p)->device_path,
                                            d->device_path);
     }
     if (ret == 0) {
       return;
     } else if (ret > 0) {
       break;
     }
   }
   ...

In the HiKey platform, I prepared the same driver for both eMMC and SD. 
So the device paths are in below.
SD: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0
cc8d514a)[9: 00 e0 23 f7 00 00 00 00 00 ]/UnknownMessaging(1a)/EndEntire
eMMC: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0
cc8d514a)[9: 00 d0 23 f7 00 00 00 00 00 
]/UnknownMessaging(1d)/Ctrl(0)/EndEntire

#define MSG_SD_DP                   0x1A
#define MSG_EMMC_DP                 0x1D

In the second level, the device paths are different.

And GRUB resort the sequence by ascending order (with above code). So SD 
device always gets higher priority than eMMC device.

If we always use installer to install OS, it may not an issue. Since 
installer could create grub.cfg by itself. But it imports another issue 
on lacking of persistent variable storage. And we need to deploy system 
without installer on embedded device.

How to fix the priority issue? Fix in GRUB or something else?

Best Regards
Haojian

Re: GRUB issue on device priority

[Haojian Zhuang]

On 2017/11/7 23:14, Vladimir 'phcoder' Serbinenko wrote:
> 
> 
> On Tue, Nov 7, 2017, 15:23 Haojian Zhuang <haojian.zhuang@linaro.org 
> <mailto:haojian.zhuang@linaro.org>> wrote:
> 
>     Hi all,
> 
>     It seems there's a device priority issue in GRUB.
> 
> Please use mailing list for grub, not just messaging random people some 
> of whom have left project 10 years ago.

Sure. Just find the mailing account. Loop grub-devel@gnu.org.

> 
> 
>     All block io handles are linked into the list in edk2, and GRUB could
>     fetch it. Then GRUB creates its own ascending on HD priority.
> 
> What do you mean by "priority"? It's just disk numbers that don't change 
> anything.

Yes, it's disk numbers and it causes an issue.

In edk2, I organize all handles in sequence. For example, eMMC is the 
first BlockIO device, and SD is the second BlockIO device.

In GRUB, it resorts the sequence by ascending order on UUID (device 
path). Since both eMMC and SD controllers are varient of one IP, use the 
same device driver for these two devices. Then UUIDs are in below. GRUB 
names SD with HD0 and eMMC with HD1. The disk numbers are inverted by GRUB.

> 
>     add_device():
>         for (p = devices; *p; p = &((*p)->next)) {
>           ret = grub_efi_compare_device_paths
>     (grub_efi_find_last_device_path
>     ((*p->device_path),
>                               
>     grub_efi_find_last_device_path->device_path));
>           if (ret == 0) {
>             ret = grub_efi_compare_device_paths ((*p)->device_path,
>                                                  d->device_path);
>           }
>           if (ret == 0) {
>             return;
>           } else if (ret > 0) {
>             break;
>           }
>         }
>         ...
> 
>     In the HiKey platform, I prepared the same driver for both eMMC and SD.
>     So the device paths are in below.
>     SD: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0
>     cc8d514a)[9: 00 e0 23 f7 00 00 00 00 00 ]/UnknownMessaging(1a)/EndEntire
>     eMMC: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0
>     cc8d514a)[9: 00 d0 23 f7 00 00 00 00 00
>     ]/UnknownMessaging(1d)/Ctrl(0)/EndEntire
> 
>     #define MSG_SD_DP                   0x1A
>     #define MSG_EMMC_DP                 0x1D
> 
>     In the second level, the device paths are different.
> 
>     And GRUB resort the sequence by ascending order (with above code). So SD
>     device always gets higher priority than eMMC device.
> 
>     If we always use installer to install OS, it may not an issue. Since
>     installer could create grub.cfg by itself. But it imports another issue
>     on lacking of persistent variable storage. And we need to deploy system
>     without installer on embedded device.
> 
>     How to fix the priority issue? Fix in GRUB or something else?
> 
> You shouldn't rely on any particular GRUB device ordering. The easiest 
> way is to use UUID.

I failed to find root by uuid. I don't know why. But I found that I can 
make use of "-hint" with multiple parameters. Then this issue could be 
fixed.

Great thanks for your help.

Best Regards
Haojian

Re: GRUB issue on device priority

[Leif Lindholm]

On Wed, Nov 08, 2017 at 01:46:13PM +0800, Haojian Zhuang wrote:
> On 2017/11/7 23:14, Vladimir 'phcoder' Serbinenko wrote:
> > On Tue, Nov 7, 2017, 15:23 Haojian Zhuang <haojian.zhuang@linaro.org
> > <mailto:haojian.zhuang@linaro.org>> wrote:
> > 
> >     Hi all,
> > 
> >     It seems there's a device priority issue in GRUB.
> > 
> > Please use mailing list for grub, not just messaging random people some
> > of whom have left project 10 years ago.
> 
> Sure. Just find the mailing account. Loop grub-devel@gnu.org.

I don't think we need to loop in grub-devel:
If you tell grub to boot from the first block device it encounters, it
will boot from the first block device it encounters. That is as
expected. I will delete grub-devel and Vladimir from the recipient
list beyond this email and then respond to your original email on
edk2-devel.

/
    Leif

Re: GRUB issue on device priority

[Leif Lindholm]

On Tue, Nov 07, 2017 at 10:02:30PM +0800, Haojian Zhuang wrote:
> Hi all,
> 
> It seems there's a device priority issue in GRUB.

GRUB is behaving as expected.

> All block io handles are linked into the list in edk2, and GRUB could fetch
> it. Then GRUB creates its own ascending on HD priority.
> add_device():
>   for (p = devices; *p; p = &((*p)->next)) {
>     ret = grub_efi_compare_device_paths (grub_efi_find_last_device_path
> ((*p->device_path),
>                          grub_efi_find_last_device_path->device_path));
>     if (ret == 0) {
>       ret = grub_efi_compare_device_paths ((*p)->device_path,
>                                            d->device_path);
>     }
>     if (ret == 0) {
>       return;
>     } else if (ret > 0) {
>       break;
>     }
>   }
>   ...

UEFI guarantees no ordering of block devices. GRUB is a UEFI
application, so it cannot assume anything with regards to device
ordering.

> In the HiKey platform, I prepared the same driver for both eMMC and
> SD. So the device paths are in below.
> SD: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 e0 23 f7 00 00 00 00 00 ]/UnknownMessaging(1a)/EndEntire
> eMMC: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 d0 23 f7 00 00 00 00 00]/UnknownMessaging(1d)/Ctrl(0)/EndEntire
> 
> #define MSG_SD_DP                   0x1A
> #define MSG_EMMC_DP                 0x1D
> 
> In the second level, the device paths are different.
> 
> And GRUB resort the sequence by ascending order (with above
> code). So SD device always gets higher priority than eMMC device.
> 
> If we always use installer to install OS, it may not an issue. Since
> installer could create grub.cfg by itself. But it imports another
> issue on lacking of persistent variable storage. And we need to
> deploy system without installer on embedded device.

Yes, this is the bit which is interesting, and why I requested you
post this problem to edk2-devel.

I believe that we need to have a sensible way to deal with embedded
platforms that do not have operating systems "installed" to them, but
rather written as images to flash devices.

So, can you clarify a few things for me?:
- Where is GRUB located in your (pre-SD) system?
- Where is GRUB located in your SD system?
- Are you looking for a way to support GRUB being located on either
  eMMC or SD? Or are you looking to always have GRUB loaded from eMMC?

/
    Leif

Re: GRUB issue on device priority

[Haojian Zhuang]

On 2017/11/8 21:53, Leif Lindholm wrote:
> On Tue, Nov 07, 2017 at 10:02:30PM +0800, Haojian Zhuang wrote:
>> Hi all,
>>
>> It seems there's a device priority issue in GRUB.
> 
> GRUB is behaving as expected.
> 
>> All block io handles are linked into the list in edk2, and GRUB could fetch
>> it. Then GRUB creates its own ascending on HD priority.
>> add_device():
>>    for (p = devices; *p; p = &((*p)->next)) {
>>      ret = grub_efi_compare_device_paths (grub_efi_find_last_device_path
>> ((*p->device_path),
>>                           grub_efi_find_last_device_path->device_path));
>>      if (ret == 0) {
>>        ret = grub_efi_compare_device_paths ((*p)->device_path,
>>                                             d->device_path);
>>      }
>>      if (ret == 0) {
>>        return;
>>      } else if (ret > 0) {
>>        break;
>>      }
>>    }
>>    ...
> 
> UEFI guarantees no ordering of block devices. GRUB is a UEFI
> application, so it cannot assume anything with regards to device
> ordering.
> 
>> In the HiKey platform, I prepared the same driver for both eMMC and
>> SD. So the device paths are in below.
>> SD: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 e0 23 f7 00 00 00 00 00 ]/UnknownMessaging(1a)/EndEntire
>> eMMC: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 d0 23 f7 00 00 00 00 00]/UnknownMessaging(1d)/Ctrl(0)/EndEntire
>>
>> #define MSG_SD_DP                   0x1A
>> #define MSG_EMMC_DP                 0x1D
>>
>> In the second level, the device paths are different.
>>
>> And GRUB resort the sequence by ascending order (with above
>> code). So SD device always gets higher priority than eMMC device.
>>
>> If we always use installer to install OS, it may not an issue. Since
>> installer could create grub.cfg by itself. But it imports another
>> issue on lacking of persistent variable storage. And we need to
>> deploy system without installer on embedded device.
> 
> Yes, this is the bit which is interesting, and why I requested you
> post this problem to edk2-devel.
> 
> I believe that we need to have a sensible way to deal with embedded
> platforms that do not have operating systems "installed" to them, but
> rather written as images to flash devices.
> 
> So, can you clarify a few things for me?:
> - Where is GRUB located in your (pre-SD) system?

In eMMC.
> - Where is GRUB located in your SD system?
In SD.
> - Are you looking for a way to support GRUB being located on either
>    eMMC or SD? Or are you looking to always have GRUB loaded from eMMC?

GRUB could be located in either in eMMC or SD.

Best Regards
Haojian

Re: GRUB issue on device priority

[Leif Lindholm]

Apologies for delay in responding.

On Thu, Nov 09, 2017 at 04:41:55PM +0800, Haojian Zhuang wrote:
> > > In the HiKey platform, I prepared the same driver for both eMMC and
> > > SD. So the device paths are in below.
> > > SD: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 e0 23 f7 00 00 00 00 00 ]/UnknownMessaging(1a)/EndEntire
> > > eMMC: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 d0 23 f7 00 00 00 00 00]/UnknownMessaging(1d)/Ctrl(0)/EndEntire
> > > 
> > > #define MSG_SD_DP                   0x1A
> > > #define MSG_EMMC_DP                 0x1D
> > > 
> > > In the second level, the device paths are different.
> > > 
> > > And GRUB resort the sequence by ascending order (with above
> > > code). So SD device always gets higher priority than eMMC device.
> > > 
> > > If we always use installer to install OS, it may not an issue. Since
> > > installer could create grub.cfg by itself. But it imports another
> > > issue on lacking of persistent variable storage. And we need to
> > > deploy system without installer on embedded device.
> > 
> > Yes, this is the bit which is interesting, and why I requested you
> > post this problem to edk2-devel.
> > 
> > I believe that we need to have a sensible way to deal with embedded
> > platforms that do not have operating systems "installed" to them, but
> > rather written as images to flash devices.
> > 
> > So, can you clarify a few things for me?:
> > - Where is GRUB located in your (pre-SD) system?
> 
> In eMMC.
>
> > - Where is GRUB located in your SD system?
>
> In SD.
>
> > - Are you looking for a way to support GRUB being located on either
> >    eMMC or SD? Or are you looking to always have GRUB loaded from eMMC?
> 
> GRUB could be located in either in eMMC or SD.

OK, so which one should be loaded in preference if both exist?

I.e., what if you write a GRUB to eMMC via USB uplink, and at some
later point you insert an uSD card?

One (hackish, but at least predictable) way of resolving this would be
to implement a PlatformBootManagerLib that never connects more than
one of those devices. Alternatively one that always disconnects the
one not used for booting.

(Yes, I always argue that we should try to use a single library across
most/all ARM platforms, but in order to figure out the useful
abstractions that would make this possible, it would be worth to have
some different versions implementing different features.)

/
    Leif