IP4 Config Troubles with DHCP

IP4 Config Troubles with DHCP

[Cohen, Eugene]

We have been running into an issue when trying to configure an interface as DHCP where if the DHCP process is not yet complete (Ip4Mode.IsConfigured is FALSE) the configure process will never succeed.

We have a case where we attempt to invoke the UDP4 Configure:

Status = Nlc->Udp4->Configure(Nlc->Udp4, &Nlc->UdpConfig);

We had a retry loop where we keep calling Udp4->Configure until we finally see Ip4Mode.IsConfigured go TRUE (similar to what you see in Mtftp4GetMapping) - this has worked for many years but recently something broke this.   Now, even when DHCP succeeds the Ip4Mode.IsConfigured flag is set to FALSE.  

Only if we retry by destroying and re-creating new service binding children can we actually get this logic to succeed.  This logic is getting ridiculously complicated so I'm thinking there has to be a better way of doing this.

Do you have an example of specifically how a driver/app should handle the case where the DHCP process is not yet complete and wants to wait?

Thanks,

Eugene

FW: IP4 Config Troubles with DHCP

[Wu, Jiaxin]

Eugene,

I want to confirm with you the steps to reproduce the issue:

1. Set policy to DHCP.
2. If DHCP process is not complete yet, then run one App to invoke the UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to TRUE     ---- failure here!!!

Above steps right? 

Actually, you don't need to retry the UDP configuration loop according the Ip4Mode.IsConfigured flag. You are only recommended to set a timer to check the mapping status after the configuration:

For example:
  Status = Nlc->Udp4->Configure(Nlc->Udp4, &Nlc->UdpConfig);
  if (EFI_ERROR (Status) && (Status != EFI_NO_MAPPING)) {
      return  Status;
  }
  if (Status == EFI_NO_MAPPING && !UdpGetMapping (Nlc->Udp4)) {
      return  Status;
  }

In UdpGetMapping () function, create one timer to check Ip4Mode.IsConfigured:

For example:
UdpGetMapping () {
  IsMapDone = FALSE;
  gBS->CreateEvent (EVT_TIMER, TPL_CALLBACK, NULL, NULL, &TimeoutEvent);
  gBS->SetTimer (TimeoutEvent, TimerRelative, AnyValue);
  while (EFI_ERROR (gBS->CheckEvent (TimeoutEvent))) {
    Udp4->Poll (Udp4);
    Udp4->GetModeData (Udp4, &Udp4Mode, & Ip4Mode, NULL, NULL);
    if (Ip4Mode.IsConfigured) {
      IsMapDone = TRUE;
      break;
    }
  }
  return IsMapDone;
}

If DHCP process succeed, Ip4Mode.IsConfigured should be updated. If not, any bug may be existed.

Thanks,
Jiaxin

-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Cohen, Eugene
Sent: Thursday, August 11, 2016 2:14 AM
To: edk2-devel@lists.01.org; Wu, Jiaxin <jiaxin.wu@intel.com>
Subject: [edk2] IP4 Config Troubles with DHCP

We have been running into an issue when trying to configure an interface as DHCP where if the DHCP process is not yet complete (Ip4Mode.IsConfigured is FALSE) the configure process will never succeed.

We have a case where we attempt to invoke the UDP4 Configure:

Status = Nlc->Udp4->Configure(Nlc->Udp4, &Nlc->UdpConfig);

We had a retry loop where we keep calling Udp4->Configure until we finally see Ip4Mode.IsConfigured go TRUE (similar to what you see in Mtftp4GetMapping) - this has worked for many years but recently something broke this.   Now, even when DHCP succeeds the Ip4Mode.IsConfigured flag is set to FALSE.  

Only if we retry by destroying and re-creating new service binding children can we actually get this logic to succeed.  This logic is getting ridiculously complicated so I'm thinking there has to be a better way of doing this.

Do you have an example of specifically how a driver/app should handle the case where the DHCP process is not yet complete and wants to wait?

Thanks,

Eugene



_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: IP4 Config Troubles with DHCP

[Cohen, Eugene]

Jianxin,

> I want to confirm with you the steps to reproduce the issue:
> 
> 1. Set policy to DHCP.
> 2. If DHCP process is not complete yet, then run one App to invoke the
> UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling
> Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
> 3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to
> TRUE     ---- failure here!!!
> 
> Above steps right?

Yes, that summarizes it well.

> Actually, you don't need to retry the UDP configuration loop according
> the Ip4Mode.IsConfigured flag. You are only recommended to set a
> timer to check the mapping status after the configuration:

This is exactly what we used to do.  Recently it stopped working which is why we now have to re-do service binding creation and configuration.

> If DHCP process succeed, Ip4Mode.IsConfigured should be updated. If
> not, any bug may be existed.

>From the traces I've looked at I can clearly see that the DHCP process as finished (lease acquired) and that Ip4Mode.IsConfigured is still FALSE.  I'll send you a detailed trace of this separately.

So yes, I think there is a bug that crept in.

Eugene

Re: IP4 Config Troubles with DHCP

[Wu, Jiaxin]

Thanks Eugene, I will try to reproduce the issue and dig it out. Any process will inform you.

Best Regards!
Jiaxin

> -----Original Message-----
> From: Cohen, Eugene [mailto:eugene@hp.com]
> Sent: Thursday, August 11, 2016 10:31 PM
> To: Wu, Jiaxin <jiaxin.wu@intel.com>; edk2-devel@lists.01.org
> Subject: RE: IP4 Config Troubles with DHCP
> 
> Jianxin,
> 
> > I want to confirm with you the steps to reproduce the issue:
> >
> > 1. Set policy to DHCP.
> > 2. If DHCP process is not complete yet, then run one App to invoke the
> > UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling
> > Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
> > 3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to
> > TRUE     ---- failure here!!!
> >
> > Above steps right?
> 
> Yes, that summarizes it well.
> 
> > Actually, you don't need to retry the UDP configuration loop according
> > the Ip4Mode.IsConfigured flag. You are only recommended to set a timer
> > to check the mapping status after the configuration:
> 
> This is exactly what we used to do.  Recently it stopped working which is why
> we now have to re-do service binding creation and configuration.
> 
> > If DHCP process succeed, Ip4Mode.IsConfigured should be updated. If
> > not, any bug may be existed.
> 
> From the traces I've looked at I can clearly see that the DHCP process as
> finished (lease acquired) and that Ip4Mode.IsConfigured is still FALSE.  I'll send
> you a detailed trace of this separately.
> 
> So yes, I think there is a bug that crept in.
> 
> Eugene

Re: IP4 Config Troubles with DHCP

[Wu, Jiaxin]

Eugene,

I can reproduce the issue now. And root cause as below:

#1. Set policy to DHCP.  
#2. If DHCP process is not complete yet, then run one App to invoke the UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
#3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to TRUE     ---- failure here!!!

In step1, the policy will be set to DHCP, and then Ip4Config2OnPolicyChanged() will be called. In this function, "IpSb->Reconfig" flag will be set to TRUE before Ip4StartAutoConfig() called. That means the original "IpSb->DefaultInterface" will be abandoned/freed once this DHCP process finished. Detailed see Ip4Config2SetDefaultAddr() function. 

In step2, UDP4 Configure with "UseDefaultAddress = TRUE" is called, that means the default interface (IpSb->DefaultInterface) will be selected as current instance's interface. Detailed see Ip4ConfigProtocol() function.

In step3, When DHCP process finished, as I said in step1, the original "IpSb->DefaultInterface" will be abandoned/freed because "IpSb->Reconfig" flag is true. Meanwhile, one new interface is assigned to "IpSb->DefaultInterface". This "IpSb->DefaultInterface" is different to the original one assigned to the UDP4 Configured instance. So, even DHCP process succeed, the up caller will never have the chance to get it's truly status. 

I will send one patch to fix this issue later.

Thanks your reporting.

Best Regards!
Jiaxin 


> -----Original Message-----
> From: Wu, Jiaxin
> Sent: Friday, August 12, 2016 8:20 AM
> To: Cohen, Eugene <eugene@hp.com>; edk2-devel@lists.01.org
> Subject: RE: IP4 Config Troubles with DHCP
> 
> Thanks Eugene, I will try to reproduce the issue and dig it out. Any process will
> inform you.
> 
> Best Regards!
> Jiaxin
> 
> > -----Original Message-----
> > From: Cohen, Eugene [mailto:eugene@hp.com]
> > Sent: Thursday, August 11, 2016 10:31 PM
> > To: Wu, Jiaxin <jiaxin.wu@intel.com>; edk2-devel@lists.01.org
> > Subject: RE: IP4 Config Troubles with DHCP
> >
> > Jianxin,
> >
> > > I want to confirm with you the steps to reproduce the issue:
> > >
> > > 1. Set policy to DHCP.
> > > 2. If DHCP process is not complete yet, then run one App to invoke
> > > the
> > > UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling
> > > Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
> > > 3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to
> > > TRUE     ---- failure here!!!
> > >
> > > Above steps right?
> >
> > Yes, that summarizes it well.
> >
> > > Actually, you don't need to retry the UDP configuration loop
> > > according the Ip4Mode.IsConfigured flag. You are only recommended to
> > > set a timer to check the mapping status after the configuration:
> >
> > This is exactly what we used to do.  Recently it stopped working which
> > is why we now have to re-do service binding creation and configuration.
> >
> > > If DHCP process succeed, Ip4Mode.IsConfigured should be updated. If
> > > not, any bug may be existed.
> >
> > From the traces I've looked at I can clearly see that the DHCP process
> > as finished (lease acquired) and that Ip4Mode.IsConfigured is still
> > FALSE.  I'll send you a detailed trace of this separately.
> >
> > So yes, I think there is a bug that crept in.
> >
> > Eugene

Re: IP4 Config Troubles with DHCP

[Cohen, Eugene]

> Eugene,
> 
> I can reproduce the issue now. And root cause as below:
> 
> #1. Set policy to DHCP.
> #2. If DHCP process is not complete yet, then run one App to invoke the
> UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling
> Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
> #3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to TRUE     --
> -- failure here!!!
> 
> In step1, the policy will be set to DHCP, and then
> Ip4Config2OnPolicyChanged() will be called. In this function, "IpSb-
> >Reconfig" flag will be set to TRUE before Ip4StartAutoConfig() called. That
> means the original "IpSb->DefaultInterface" will be abandoned/freed once
> this DHCP process finished. Detailed see Ip4Config2SetDefaultAddr()
> function.
> 
> In step2, UDP4 Configure with "UseDefaultAddress = TRUE" is called, that
> means the default interface (IpSb->DefaultInterface) will be selected as
> current instance's interface. Detailed see Ip4ConfigProtocol() function.
> 
> In step3, When DHCP process finished, as I said in step1, the original "IpSb-
> >DefaultInterface" will be abandoned/freed because "IpSb->Reconfig" flag is
> true. Meanwhile, one new interface is assigned to "IpSb->DefaultInterface".
> This "IpSb->DefaultInterface" is different to the original one assigned to the
> UDP4 Configured instance. So, even DHCP process succeed, the up caller will
> never have the chance to get it's truly status.
> 
> I will send one patch to fix this issue later.
> 
> Thanks your reporting.
> 
> Best Regards!
> Jiaxin

Jiaxin, excellent to hear.  I'm glad to hear you've isolated the issue.  So the service binding instance we have in this case is orphaned which explains why destroying it and creating a new one resolves the issue.  Of course we'd be happy to test the patch as soon as it's available.

Thank you!

Eugene

Re: IP4 Config Troubles with DHCP

[Wu, Jiaxin]

Hi Eugene,

The formal patch will be sent out next Monday. Currently, I'm not in office, so, I don't have valid DHCP server to verify my fixing. 

Thank you for understanding.

Jiaxin 

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Cohen, Eugene
> Sent: Friday, August 12, 2016 8:35 PM
> To: Wu, Jiaxin <jiaxin.wu@intel.com>; edk2-devel@lists.01.org
> Subject: Re: [edk2] IP4 Config Troubles with DHCP
> 
> > Eugene,
> >
> > I can reproduce the issue now. And root cause as below:
> >
> > #1. Set policy to DHCP.
> > #2. If DHCP process is not complete yet, then run one App to invoke
> > the
> > UDP4 Configure with "UseDefaultAddress = TRUE" (loop to keep calling
> > Udp4->Configure until Ip4Mode.IsConfigured changes to TRUE)
> > #3. Even DHCP succeed but Ip4Mode.IsConfigured flag never set to TRUE
> --
> > -- failure here!!!
> >
> > In step1, the policy will be set to DHCP, and then
> > Ip4Config2OnPolicyChanged() will be called. In this function, "IpSb-
> > >Reconfig" flag will be set to TRUE before Ip4StartAutoConfig()
> > >called. That
> > means the original "IpSb->DefaultInterface" will be abandoned/freed
> > once this DHCP process finished. Detailed see
> > Ip4Config2SetDefaultAddr() function.
> >
> > In step2, UDP4 Configure with "UseDefaultAddress = TRUE" is called,
> > that means the default interface (IpSb->DefaultInterface) will be
> > selected as current instance's interface. Detailed see Ip4ConfigProtocol()
> function.
> >
> > In step3, When DHCP process finished, as I said in step1, the original
> > "IpSb-
> > >DefaultInterface" will be abandoned/freed because "IpSb->Reconfig"
> > >flag is
> > true. Meanwhile, one new interface is assigned to "IpSb-
> >DefaultInterface".
> > This "IpSb->DefaultInterface" is different to the original one
> > assigned to the
> > UDP4 Configured instance. So, even DHCP process succeed, the up caller
> > will never have the chance to get it's truly status.
> >
> > I will send one patch to fix this issue later.
> >
> > Thanks your reporting.
> >
> > Best Regards!
> > Jiaxin
> 
> Jiaxin, excellent to hear.  I'm glad to hear you've isolated the issue.  So the
> service binding instance we have in this case is orphaned which explains why
> destroying it and creating a new one resolves the issue.  Of course we'd be
> happy to test the patch as soon as it's available.
> 
> Thank you!
> 
> Eugene
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: IP4 Config Troubles with DHCP

[Cohen, Eugene]

Jiaxin,
 
> Actually, you don't need to retry the UDP configuration loop according
> the Ip4Mode.IsConfigured flag. You are only recommended to set a
> timer to check the mapping status after the configuration:
> 
> For example:
>   Status = Nlc->Udp4->Configure(Nlc->Udp4, &Nlc->UdpConfig);
>   if (EFI_ERROR (Status) && (Status != EFI_NO_MAPPING)) {
>       return  Status;
>   }
>   if (Status == EFI_NO_MAPPING && !UdpGetMapping (Nlc->Udp4)) {
>       return  Status;
>   }
> 
> In UdpGetMapping () function, create one timer to check
> Ip4Mode.IsConfigured:
> 
> For example:
> UdpGetMapping () {
>   IsMapDone = FALSE;
>   gBS->CreateEvent (EVT_TIMER, TPL_CALLBACK, NULL, NULL,
> &TimeoutEvent);
>   gBS->SetTimer (TimeoutEvent, TimerRelative, AnyValue);
>   while (EFI_ERROR (gBS->CheckEvent (TimeoutEvent))) {
>     Udp4->Poll (Udp4);
>     Udp4->GetModeData (Udp4, &Udp4Mode, & Ip4Mode, NULL,
> NULL);
>     if (Ip4Mode.IsConfigured) {
>       IsMapDone = TRUE;
>       break;
>     }
>   }
>   return IsMapDone;
> }
> 
> If DHCP process succeed, Ip4Mode.IsConfigured should be updated. If
> not, any bug may be existed.

In testing the new patch (removing RECONFIG=TRUE) I see that the statement you made above is not accurate when the protocol is TCP.  When Configure is called the first time it returns EFI_NO_MAPPING.  This seems to be remembered in the socket state: ((Sock)->ConfigureState == SO_NO_MAPPING) so that any attempt to use the instance after Ip4Mode.IsConfigured goes TRUE fails (like for a TCP4 Listen).

So for TCP we must issue another Configure request to clean up this state so it's not as simple as just polling the GetModeData result, at least for TCP.

Do you believe this is expected behavior?

Thanks,

Eugene

Re: IP4 Config Troubles with DHCP

[Wu, Jiaxin]

> > Actually, you don't need to retry the UDP configuration loop according
> > the Ip4Mode.IsConfigured flag. You are only recommended to set a timer
> > to check the mapping status after the configuration:
> >
> > For example:
> >   Status = Nlc->Udp4->Configure(Nlc->Udp4, &Nlc->UdpConfig);
> >   if (EFI_ERROR (Status) && (Status != EFI_NO_MAPPING)) {
> >       return  Status;
> >   }
> >   if (Status == EFI_NO_MAPPING && !UdpGetMapping (Nlc->Udp4)) {
> >       return  Status;
> >   }
> >
> > In UdpGetMapping () function, create one timer to check
> > Ip4Mode.IsConfigured:
> >
> > For example:
> > UdpGetMapping () {
> >   IsMapDone = FALSE;
> >   gBS->CreateEvent (EVT_TIMER, TPL_CALLBACK, NULL, NULL,
> > &TimeoutEvent);
> >   gBS->SetTimer (TimeoutEvent, TimerRelative, AnyValue);
> >   while (EFI_ERROR (gBS->CheckEvent (TimeoutEvent))) {
> >     Udp4->Poll (Udp4);
> >     Udp4->GetModeData (Udp4, &Udp4Mode, & Ip4Mode, NULL, NULL);
> >     if (Ip4Mode.IsConfigured) {
> >       IsMapDone = TRUE;
> >       break;
> >     }
> >   }
> >   return IsMapDone;
> > }
> >
> > If DHCP process succeed, Ip4Mode.IsConfigured should be updated. If
> > not, any bug may be existed.
> 
> In testing the new patch (removing RECONFIG=TRUE) I see that the
> statement you made above is not accurate when the protocol is TCP.  When
> Configure is called the first time it returns EFI_NO_MAPPING.  This seems to
> be remembered in the socket state: ((Sock)->ConfigureState ==
> SO_NO_MAPPING) so that any attempt to use the instance after
> Ip4Mode.IsConfigured goes TRUE fails (like for a TCP4 Listen).
> 
> So for TCP we must issue another Configure request to clean up this state so
> it's not as simple as just polling the GetModeData result, at least for TCP.

Yeah. When I was drafting the UDP APP to test the new fix, I found the same case you mentioned. We must issue another UDP Configure () to clean the previous state once the  Ip4Mode.IsConfigured is TRUE. So, the above example is not accurate with my the current implementation:(. But I'm still not recommend to loop the UDP configuration every time if Ip4Mode.IsConfigured is false. The right behavior for UDP/TCP is 1) timer check the Ip4Mode.IsConfigured, 2) Once Ip4Mode.IsConfigured is TRUE, reconfigure the instance again. Sorry for the above example was troubling you. Also use UDP as example, correct as below:

    Udata.UseDefaultAddress = TRUE;
    Status = Udp4->Configure (Udp4, &Udata);
    if (EFI_ERROR (Status) && (Status != EFI_NO_MAPPING)) {
      return Status;
    }
    if ((Status == EFI_NO_MAPPING) && !UdpGetMapping (Udp4, &Udata)) {
      return Status;
    }

In UdpGetMapping () function:

BOOLEAN
UdpGetMapping (
  IN EFI_UDP4_PROTOCOL  *Udp4,
  IN EFI_UDP4_CONFIG_DATA   *UdpCfgData
  )
{
  EFI_STATUS          Status;
  EFI_EVENT            TimeoutEvent;
  EFI_IP4_MODE_DATA    Ip4ModeData;
  BOOLEAN              IsMapDone;

  IsMapDone = FALSE;

  ZeroMem (&Ip4ModeData, sizeof (EFI_IP4_MODE_DATA));

  Status = gBS->CreateEvent (EVT_TIMER, TPL_CALLBACK, NULL, NULL, &TimeoutEvent);
  if (EFI_ERROR (Status)) {
    return IsMapDone;
  }

  //
  // Start the timer, it will timeout after 5 seconds.
  //
  gBS->SetTimer (TimeoutEvent, TimerRelative, 500000000);

  while (EFI_ERROR (gBS->CheckEvent (TimeoutEvent))) {
    Udp4->Poll (Udp4);

    if (!EFI_ERROR (Udp4->GetModeData (Udp4, NULL, &Ip4ModeData, NULL, NULL)) &&
        Ip4ModeData.IsConfigured) {
      Udp4->Configure (Udp4, NULL);
      IsMapDone = (BOOLEAN) (Udp4->Configure (Udp4, UdpCfgData) == EFI_SUCCESS);
      break;
    }
  }

  gBS->CloseEvent (TimeoutEvent);

  return IsMapDone;
}

> 
> Do you believe this is expected behavior?
> 
> Thanks,
> 
> Eugene
> 

Re: IP4 Config Troubles with DHCP

[Cohen, Eugene]

Jiaxin,

> Yeah. When I was drafting the UDP APP to test the new fix, I found the
> same case you mentioned. We must issue another UDP Configure () to
> clean the previous state once the  Ip4Mode.IsConfigured is TRUE. So,
> the above example is not accurate with my the current
> implementation:(. But I'm still not recommend to loop the UDP
> configuration every time if Ip4Mode.IsConfigured is false. The right
> behavior for UDP/TCP is 1) timer check the Ip4Mode.IsConfigured, 2)
> Once Ip4Mode.IsConfigured is TRUE, reconfigure the instance again.
> Sorry for the above example was troubling you. Also use UDP as
> example, correct as below:

Can't we just call this a defect and make it so the first Configure() that returns IsConfigured=TRUE works?  It seems much safer to handle this in the stack than to expect hundreds or thousands of different network applications and services to try to implement this sequence correctly.

I don't see where in the UEFI spec it states that you must call Configure(cfg) Configure(NULL) Configure(cfg) just to make it work...

Thanks,

Eugene