Adding OpenSSL as the submodule of EDKII project...

Adding OpenSSL as the submodule of EDKII project...

[Long, Qin]

Hi,

The Git submodule allows us to keep another Git repository in a subdirectory of main project. The Submodule repository has its own history, which does not interfere with the history of the current repository. This can be used to have external dependencies such as third party libraries.

After the extra patch for EDKII-OpenSSL build was removed, OpenSSL can be one typical use case of Git Submodule in EDKII project. The Git parent (EDKII) will keep track of the release version / tag IDs of Submodules when the module owner commit. That will also help to ensure that when we check out the EDKII project then the openssl Submodule will also contain its right tags.

One forked EDK2 repository with OpenSSL submodule support was available at https://github.com/qloong/edk2 for testing.

For EDKII developers, the possible impacts will include (comparing to the original openssl source download / unpacking mechanism):

-          Cloning EDKII project with Submodules
The user can use the following commands to clone both main EDKII repo and openssl submodule:
1) Add the "--recursive" flag to their git clone command:
      $ git clone --recursive https://github.com/qloong/edk2
or
2) Manually initialize and update the submodules after the clone operation on main project:
      $ git clone https://github.com/qloong/edk2
      $ git submodule update -init -recursive


-          Pulling in Upstream Changes
For Pull operations, one single "git pull" will not update the submodule repository. So the following combined commands can be used to pull the remote submodule
updates (e.g. updating to new supported OpenSSL release tag)
$ git pull -recurse-submodules && git submodule update -recursive -remote

(For any third-party GUI tools (e.g. TortoiseGit), there are also no direct support to sync-up the primary and submodule repo. We need to use extra "Pull..." and "Submodule Update..." to handle this case.)

Let me know your comments & suggestions on this possible submodule updates (advantage or disadvantage of this change? Any impacts? ...). Thanks.


Best Regards & Thanks,
LONG, Qin

Re: Adding OpenSSL as the submodule of EDKII project...

[Sean Brogan]

Long,Qin

I think this is a great idea and great step forward for making edk2 more consumable.  We already do this for our internal clones of edk2 and would like to see more work like this done to make edk2 consumable in a sustainable and easy way.  For example we also use submodules within our clone of edk2 for win32 basetools bin and nasm.  We then use submodules exclusively to manage consuming edk2 into our project repos.  This gives us great flexibility and agility to manage, update, and sustain our code trees.  TianoCore should think of itself not as the final repo but as an ingredient in a larger repository for building and shipping UEFI based products.   In that end I would like to see EDK2 break into smaller repositories (but I'll save that for another day).  

Thanks
Sean


-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Long, Qin
Sent: Thursday, July 20, 2017 12:18 AM
To: edk2-devel@lists.01.org
Cc: Long, Qin <qin.long@intel.com>
Subject: [edk2] Adding OpenSSL as the submodule of EDKII project...

Hi,

The Git submodule allows us to keep another Git repository in a subdirectory of main project. The Submodule repository has its own history, which does not interfere with the history of the current repository. This can be used to have external dependencies such as third party libraries.

After the extra patch for EDKII-OpenSSL build was removed, OpenSSL can be one typical use case of Git Submodule in EDKII project. The Git parent (EDKII) will keep track of the release version / tag IDs of Submodules when the module owner commit. That will also help to ensure that when we check out the EDKII project then the openssl Submodule will also contain its right tags.

One forked EDK2 repository with OpenSSL submodule support was available at https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fqloong%2Fedk2&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=JlxveSrNDXpxECNOAS7zxfDkOvz%2FX5rc9RE%2FA9XYroA%3D&reserved=0 for testing.

For EDKII developers, the possible impacts will include (comparing to the original openssl source download / unpacking mechanism):

-          Cloning EDKII project with Submodules
The user can use the following commands to clone both main EDKII repo and openssl submodule:
1) Add the "--recursive" flag to their git clone command:
      $ git clone --recursive https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fqloong%2Fedk2&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=JlxveSrNDXpxECNOAS7zxfDkOvz%2FX5rc9RE%2FA9XYroA%3D&reserved=0
or
2) Manually initialize and update the submodules after the clone operation on main project:
      $ git clone https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fqloong%2Fedk2&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=JlxveSrNDXpxECNOAS7zxfDkOvz%2FX5rc9RE%2FA9XYroA%3D&reserved=0
      $ git submodule update -init -recursive


-          Pulling in Upstream Changes
For Pull operations, one single "git pull" will not update the submodule repository. So the following combined commands can be used to pull the remote submodule updates (e.g. updating to new supported OpenSSL release tag) $ git pull -recurse-submodules && git submodule update -recursive -remote

(For any third-party GUI tools (e.g. TortoiseGit), there are also no direct support to sync-up the primary and submodule repo. We need to use extra "Pull..." and "Submodule Update..." to handle this case.)

Let me know your comments & suggestions on this possible submodule updates (advantage or disadvantage of this change? Any impacts? ...). Thanks.


Best Regards & Thanks,
LONG, Qin
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.01.org%2Fmailman%2Flistinfo%2Fedk2-devel&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=SU3grvQcPSSDREJnEvg%2F6m55JCXJV01jAoSZi2nwcZA%3D&reserved=0

Re: Adding OpenSSL as the submodule of EDKII project...

[Long, Qin]

Sean,

Thank you for the comments.
And for the submodule maintenance, do you have any BKMs (e.g. alias setting, any scripts for synchronous clone/pull...) for sharing? 


Best Regards & Thanks,
LONG, Qin

-----Original Message-----
From: Sean Brogan [mailto:sean.brogan@microsoft.com] 
Sent: Saturday, July 22, 2017 12:52 AM
To: Long, Qin <qin.long@intel.com>; edk2-devel@lists.01.org
Subject: RE: Adding OpenSSL as the submodule of EDKII project...

Long,Qin

I think this is a great idea and great step forward for making edk2 more consumable.  We already do this for our internal clones of edk2 and would like to see more work like this done to make edk2 consumable in a sustainable and easy way.  For example we also use submodules within our clone of edk2 for win32 basetools bin and nasm.  We then use submodules exclusively to manage consuming edk2 into our project repos.  This gives us great flexibility and agility to manage, update, and sustain our code trees.  TianoCore should think of itself not as the final repo but as an ingredient in a larger repository for building and shipping UEFI based products.   In that end I would like to see EDK2 break into smaller repositories (but I'll save that for another day).  

Thanks
Sean


-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Long, Qin
Sent: Thursday, July 20, 2017 12:18 AM
To: edk2-devel@lists.01.org
Cc: Long, Qin <qin.long@intel.com>
Subject: [edk2] Adding OpenSSL as the submodule of EDKII project...

Hi,

The Git submodule allows us to keep another Git repository in a subdirectory of main project. The Submodule repository has its own history, which does not interfere with the history of the current repository. This can be used to have external dependencies such as third party libraries.

After the extra patch for EDKII-OpenSSL build was removed, OpenSSL can be one typical use case of Git Submodule in EDKII project. The Git parent (EDKII) will keep track of the release version / tag IDs of Submodules when the module owner commit. That will also help to ensure that when we check out the EDKII project then the openssl Submodule will also contain its right tags.

One forked EDK2 repository with OpenSSL submodule support was available at https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fqloong%2Fedk2&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=JlxveSrNDXpxECNOAS7zxfDkOvz%2FX5rc9RE%2FA9XYroA%3D&reserved=0 for testing.

For EDKII developers, the possible impacts will include (comparing to the original openssl source download / unpacking mechanism):

-          Cloning EDKII project with Submodules
The user can use the following commands to clone both main EDKII repo and openssl submodule:
1) Add the "--recursive" flag to their git clone command:
      $ git clone --recursive https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fqloong%2Fedk2&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=JlxveSrNDXpxECNOAS7zxfDkOvz%2FX5rc9RE%2FA9XYroA%3D&reserved=0
or
2) Manually initialize and update the submodules after the clone operation on main project:
      $ git clone https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fqloong%2Fedk2&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=JlxveSrNDXpxECNOAS7zxfDkOvz%2FX5rc9RE%2FA9XYroA%3D&reserved=0
      $ git submodule update -init -recursive


-          Pulling in Upstream Changes
For Pull operations, one single "git pull" will not update the submodule repository. So the following combined commands can be used to pull the remote submodule updates (e.g. updating to new supported OpenSSL release tag) $ git pull -recurse-submodules && git submodule update -recursive -remote

(For any third-party GUI tools (e.g. TortoiseGit), there are also no direct support to sync-up the primary and submodule repo. We need to use extra "Pull..." and "Submodule Update..." to handle this case.)

Let me know your comments & suggestions on this possible submodule updates (advantage or disadvantage of this change? Any impacts? ...). Thanks.


Best Regards & Thanks,
LONG, Qin
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.01.org%2Fmailman%2Flistinfo%2Fedk2-devel&data=02%7C01%7Csean.brogan%40microsoft.com%7C25cfc5e585ab4364560708d4cf3f860c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636361319131460644&sdata=SU3grvQcPSSDREJnEvg%2F6m55JCXJV01jAoSZi2nwcZA%3D&reserved=0