From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by mx.groups.io with SMTP id smtpd.web11.1322.1659380139146813747 for ; Mon, 01 Aug 2022 11:55:39 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20210112 header.b=XWoja6OH; spf=pass (domain: gmail.com, ip: 209.85.216.53, mailfrom: ayushdevel1325@gmail.com) Received: by mail-pj1-f53.google.com with SMTP id q7-20020a17090a7a8700b001f300db8677so13008536pjf.5 for ; Mon, 01 Aug 2022 11:55:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:from:to:cc; bh=dz8c3erIfHEXsaOnBXLgSNqjZ/1BzrW/TfnikrSNPes=; b=XWoja6OHBIeIMQBm+yFHcYdRIs1D+iBCHo4Zr+fLuLjisxxaUBTAydhIl+hrTdwvJX BRMfCSYKv82bnh38c4OwjlKJS0QeV55MP1RN8PD0oZB8hHomLgG+hh95H4pQtQ9DZVOu btmmnH+Pycxtd5BklIZWcTbrKasH0cL5rEowjT4SGYJuXeTSOpR3NoTPbzDSua4f9Rjs Q46ykLmmySGwNDRNXa8cyUeVK8fbxWZZf8oGunFki3q68HvPD0uAV3zdKUkRb0YdLTDS zDa2a00Vn7JqQoP6WYh0uPMBanCXZRMjzyLB0BDudvqbTVHIrX4OZC60xcFh0f0Sjx1K 1FcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc; bh=dz8c3erIfHEXsaOnBXLgSNqjZ/1BzrW/TfnikrSNPes=; b=3Ewn9PF7wleBwOWlYJ8iZkRLuQ6qHSbvwF9LWVirvnSdeeXhLevfJEMJyCyELJ2MKw AMrUCW9K8hsz/CrzdmfUgNY3EuEZpYmzQFP3M1VhpmvV8VjSE786dklKduCHqYcHJHSK kJh9M3yRGaRbz3o8QqWWjAP2VzZp1kSNFBQQU23iveL5LVw67VGheqcXLZAoYUbJKVGQ 9vkgcot7TK0gPauVfdq5sWGSFyG7iavhqoyOEnA4kg67I/kp5MNUQwvAPwa+mIs80wlq FReEDeGEcuS33h3IOBrD6RTmarG6FKljSg+N5ENzeU7u+YlbuWbCImVlbq/HulQ+wQek bnhA== X-Gm-Message-State: ACgBeo0LQ9fZF4skUgNChHHAmmWoo8xyiRrVfruJKHYtJyXY4dcETpeM /hPNlKV9mPSUnGGQizDaV+oNxBZXK/o= X-Google-Smtp-Source: AA6agR7y1hyEckbTisQgauv8KCEoGEh98ymXjiZLNBYPKqHYFGAz5AUmBBGva8/8D8TWCSnvmjB3fw== X-Received: by 2002:a17:902:ab90:b0:16d:c280:3c27 with SMTP id f16-20020a170902ab9000b0016dc2803c27mr17734488plr.58.1659380137885; Mon, 01 Aug 2022 11:55:37 -0700 (PDT) Return-Path: Received: from ?IPV6:2401:4900:1f3e:17ce:2380:94ae:b852:be4? ([2401:4900:1f3e:17ce:2380:94ae:b852:be4]) by smtp.gmail.com with ESMTPSA id s18-20020a170902ea1200b0016c28fbd7e5sm10226564plg.268.2022.08.01.11.55.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 01 Aug 2022 11:55:37 -0700 (PDT) Message-ID: <0ed16028-ef43-49ce-9b1c-10cb6ab656e3@gmail.com> Date: Tue, 2 Aug 2022 00:25:32 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [edk2-devel] Proposal to move Rust std work to a Repository under Tianocore To: devel@edk2.groups.io, jiewen.yao@intel.com, Pedro Falcato Cc: "Kinney, Michael D" , Michael Kubacki , "Gaibusab, Jabeena B" References: <33ac1f75-d6c9-2093-8701-698a3e553d64@gmail.com> <40ce2f41-b08d-0f26-cefe-d5805959626b@gmail.com> From: "Ayush Singh" In-Reply-To: Content-Type: multipart/alternative; boundary="------------IBlh3E3LrjzzbhTgFfXuKwKJ" Content-Language: en-US --------------IBlh3E3LrjzzbhTgFfXuKwKJ Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi Yao Jiewen. The currently implemented portions in the `std::sys` module would have to be the following: - [x] alloc   - [x] GlobalAlloc: Currently uses hardcoded `EFI_MEMORY_TYPE::EfiLoaderData`. Can be changed. - [x] args - [x] cmath: Provided by `compiler_builtins` for UEFI. - [x] env: Just contains some constants - [ ] fs   - [ ] File: Works for file on any volume as long as path can be converted to DEVICE_PATH_PROTOCOL     - [x] open     - [x] file_attr     - [x] *fsync: Uses `EFI_FILE_PROTOCOL.Flush()` internally     - [x] *datasync: Uses fsync internally     - [x] truncate     - [x] read     - [x] *read_vectored: Using rust default implementation.     - [x] is_read_vectored     - [x] write     - [x] *write_vectored: Using rust default implementation.     - [x] is_write_vectored     - [x] flush: Don't really maintain any buffer in Rust side.     - [x] seek     - [ ] duplicate     - [x] set_permissions   - [x] FileAttr   - [x] ReadDir   - [x] DirEntry   - [x] OpenOptions   - [x] FilePermissions   - [x] FileType     - [x] *is_symlink: Just returns false since I don't think UEFI supports symlinks.   - [x] DirBuilder   - [x] readdir   - [x] unlink   - [x] rename   - [x] set_perm   - [x] rmdir   - [x] remove_dir_all   - [x] try_exists   - [ ] readlink   - [ ] symlink   - [ ] link   - [x] stat   - [x] lstat   - [ ] canonicalize   - [ ] copy - [x] *io: Using the default implementation at `unsupported/io.rs`. It works fine. - [x] *locks: Using the default implementation at `unsupported/locks.rs`. It should work for any target having just a single-thread. - [ ] *net: Only implmented TCPv4 right now.   - [ ] TcpStream     - [x] connect     - [ ] connect_timeout     - [ ] set_read_timeout     - [ ] set_write_timeout     - [ ] read_timeout     - [ ] write_timeout     - [ ] peek     - [x] read     - [x] read_vectored     - [x] is_read_vectored     - [x] write     - [x] write_vectored     - [x] is_write_vectored     - [x] peer_addr     - [x] socket_addr     - [ ] *shutdown:  Only implemented complete shutdown right now.     - [ ] duplicate     - [ ] linger     - [ ] set_nodelay     - [x] nodelay     - [ ] set_ttl     - [x] ttl     - [ ] take_error     - [ ] set_nonblocking   - [ ] TcpListener     - [x] bind     - [ ] socket_addr     - [x] accept     - [ ] duplicate     - [ ] set_ttl     - [x] ttl     - [ ] set_only_v6     - [ ] only_v6     - [ ] take_error     - [ ] set_nonblocking   - [ ] UdpSocket - [ ] os   - [x] errno   - [x] error_string   - [x] getcwd: Returns the Text representation of `EFI_DEVICE_PATH_PROTOCOL`. This can be directly converted to `EFI_DEVICE_PATH_PROTOCOL` using the `EFI_DEVICE_PATH_FROM_TEXT_PROTOCOL`.   - [ ] chdir   - [ ] SplitPaths   - [ ] split_paths   - [ ] JoinPaths   - [ ] join_paths   - [x] current_exe: Returns the Text representation of `EFI_DEVICE_PATH_PROTOCOL`. This can be directly converted to `EFI_DEVICE_PATH_PROTOCOL` using the `EFI_DEVICE_PATH_FROM_TEXT_PROTOCOL`.   - [ ] Env   - [ ] env   - [x] *getenv: Currently using a static Guid. Maybe should use the Guid used by UefiShell for Environment Variables.   - [x] setenv   - [x] unsetenv   - [ ] temp_dir   - [ ] home_dir   - [x] exit   - [ ] getpid - [x] os_str: Basically just UTF-8 strings. This is because os_str is supposed to be the superset of both the OS specific string(UCS-2) and Rust strings (UTF-8).   - [x] Buf   - [x] Slice - [x] path - [ ] pipe   - [ ] *AnonPipe: Implemented using Runtime Variables.     - [x] read     - [x] read_vectored: Using default implementation     - [x] is_read_vectored     - [x] write     - [x] write_vectored: Using default implementation     - [x] is_write_vectored     - [ ] diverge   - [x] *read2: It is blocking and synchronous right now - [ ] process   - [ ] Command     - [x] new     - [x] arg     - [x] env_mut     - [ ] cwd     - [ ] *stdin: Only null working yet.     - [x] *stdout: Using my primitive AnonPipe     - [x] *stderr: Using my primitive AnonPipe     - [x] get_program     - [x] get_args     - [x] get_envs     - [x] get_current_dir     - [x] *spawn: Currently calling `EFI_BOOT_SERVICES.StartImage()` here since the pipes don't really work asynchronously right now.   - [x] StdioPipes   - [x] Stdio   - [x] ExitStatus     - [x] exit_ok     - [x] code   - [x] ExitStatusError     - [x] code   - [x] ExitCode     - [x] as_i32   - [ ] Process     - [ ] id     - [ ] kill     - [x] wait     - [ ] try_wait   - [ ] CommandArgs - [x] stdio   - [x] *STDIN_BUF_SIZE: Currently using the same value as Windows   - [x] Stdin     - [x] *read: Buffered reading not implemented yet.   - [x] Stdout   - [x] Stderr - [ ] thread     - [x] sleep - [ ] thread_local_key: Using `unsupported/thread_local_key.rs` - [x] time   - [x] Instant: Implemented same as `SystemTime`   - [x] SystemTime: Implemented using `GetTime()` All the tests for Global Allocator pass as well, so all the alloc types should work without any problem. The most fragile portions are probably `process` and `net` module. They were basically implemented for running the tests and so, only the portions that are required were implemented. Finally, the whole `libtest` mostly works, although the stdio capturing with ||||`panic-abort-tests` option is broken (the tests do output to stdio correctly). As for the up-streaming, the requirements aren't all that strict (since its a Tier-3 target). The current implementation should technically be enough to at least open a PR. I would like to fix a few things before opening the PR (and add the documentation of what Protocols the different portions of std use). I have been in touch with the UEFI Rust maintainer @David Rheinsberg, so it should be straightforward to get the review process going. The PR getting merged isn't really a GSoC target (although it would certainly be great). However, getting the project to a state, where the major hurdles for merging and usability are overcome is certainly a goal. Also, I am planning on completing the merge (and possibly helping future UEFI contributions upstream), even after GSoC duration is over. Ayush Singh On 8/1/22 23:37, Yao, Jiewen wrote: > > Hi Ayush > > Thanks for the great work. > > Would you please help me understand that how far we are between > current [1] and final upstream to Rust? > > Is upstreaming a goal of GSoC project? > > Thank you > > Yao Jiewen > > *From:* devel@edk2.groups.io *On Behalf Of > *Ayush Singh > *Sent:* Tuesday, August 2, 2022 1:50 AM > *To:* Pedro Falcato ; edk2-devel-groups-io > > *Cc:* Kinney, Michael D ; Michael Kubacki > ; Yao, Jiewen ; > Gaibusab, Jabeena B > *Subject:* Re: [edk2-devel] Proposal to move Rust std work to a > Repository under Tianocore > > Hi Pedro. To clarify the original email. The proposal is not to create > a new repository to start Rust std work. Rather, it is to move all the > per-existing work that I have done for implementing std since the > beginning of GSoC. This work can be found in my personal fork [1]. A > significant portion of std is already in a working state for DXE UEFI > and is at a point that a PR can be opened in a few weeks upstream to > get it merged. A fork under Tianocore would allow more people, form > both Rust and Tianocore side to experiment/improve the std, with the > final goal of getting it all merged in upstream Rust. > > Yours Sincerely > > Ayush Singh > > [1]: https://github.com/Ayush1325/rust/tree/uefi-std-rebase > > On 8/1/22 22:56, Pedro Falcato wrote: > > Hi, > > May I suggest you just port the bare rust language (no crates, no > std) to EDK2? It seems far more plausible to expect people to use > a cut down version with some bindings to the rest of the project > instead of hoping people just use the whole of rust, a lot of > which isnt proven (or even used AFAIK) in bare metal projects. > Porting just the bare minimum is way more realistic in my opinion. > > Thanks, > > Pedro > > On Mon, 1 Aug 2022, 18:02 Ayush Singh, > wrote: > > Hello everyone. In the previous email thread [1], I discussed the > proposal to move Rust std work to edk2-staging and mentioned its > potential problems. After some discussion with mentors, we > arrived at > the conclusion to have a rustlang [2] fork under the Tianocore > organization, and move all the std related work there. We can > then open > a PR upstream from there, while allowing PRs in this > repository. This > should help provide an easier and streamlined way for people to > experiment and work on this project while it is in the process > of being > merged upstream. > > > For a status update about tests: > > - passed: 12797 > > - failed: 40 > > - ignored: 375 > > > Yours Sincerely, > > Ayush Singh > > > [1]: https://edk2.groups.io/g/devel/message/91989 > > [2]: https://github.com/rust-lang/rust > > > > > > --------------IBlh3E3LrjzzbhTgFfXuKwKJ Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hi Yao Jiewen. The currently implemented portions in the `std::sys` module would have to be the following:

- [x] alloc
  - [x] GlobalAlloc: Currently uses hardcoded `EFI_MEMORY_TYPE::EfiLoaderData`. Can be changed.
- [x] args
- [x] cmath: Provided by `compiler_builtins` for UEFI.
- [x] env: Just contains some constants
- [ ] fs
  - [ ] File: Works for file on any volume as long as path can be converted to DEVICE_PATH_PROTOCOL
    - [x] open
    - [x] file_attr
    - [x] *fsync: Uses `EFI_FILE_PROTOCOL.Flush()` internally
    - [x] *datasync: Uses fsync internally
    - [x] truncate
    - [x] read
    - [x] *read_vectored: Using rust default implementation.
    - [x] is_read_vectored
    - [x] write
    - [x] *write_vectored: Using rust default implementation.
    - [x] is_write_vectored
    - [x] flush: Don't really maintain any buffer in Rust side.
    - [x] seek
    - [ ] duplicate
    - [x] set_permissions
  - [x] FileAttr
  - [x] ReadDir
  - [x] DirEntry
  - [x] OpenOptions
  - [x] FilePermissions
  - [x] FileType
    - [x] *is_symlink: Just returns false since I don't think UEFI supports symlinks.
  - [x] DirBuilder
  - [x] readdir
  - [x] unlink
  - [x] rename
  - [x] set_perm
  - [x] rmdir
  - [x] remove_dir_all
  - [x] try_exists
  - [ ] readlink
  - [ ] symlink
  - [ ] link
  - [x] stat
  - [x] lstat
  - [ ] canonicalize
  - [ ] copy
- [x] *io: Using the default implementation at `unsupported/io.rs`. It works fine.
- [x] *locks: Using the default implementation at `unsupported/locks.rs`. It should work for any target having just a single-thread.
- [ ] *net: Only implmented TCPv4 right now.
  - [ ] TcpStream
    - [x] connect
    - [ ] connect_timeout
    - [ ] set_read_timeout
    - [ ] set_write_timeout
    - [ ] read_timeout
    - [ ] write_timeout
    - [ ] peek
    - [x] read
    - [x] read_vectored
    - [x] is_read_vectored
    - [x] write
    - [x] write_vectored
    - [x] is_write_vectored
    - [x] peer_addr
    - [x] socket_addr
    - [ ] *shutdown:  Only implemented complete shutdown right now.
    - [ ] duplicate
    - [ ] linger
    - [ ] set_nodelay
    - [x] nodelay
    - [ ] set_ttl
    - [x] ttl
    - [ ] take_error
    - [ ] set_nonblocking
  - [ ] TcpListener
    - [x] bind
    - [ ] socket_addr
    - [x] accept
    - [ ] duplicate
    - [ ] set_ttl
    - [x] ttl
    - [ ] set_only_v6
    - [ ] only_v6
    - [ ] take_error
    - [ ] set_nonblocking
  - [ ] UdpSocket
- [ ] os
  - [x] errno
  - [x] error_string
  - [x] getcwd: Returns the Text representation of `EFI_DEVICE_PATH_PROTOCOL`. This can be directly converted to `EFI_DEVICE_PATH_PROTOCOL` using the `EFI_DEVICE_PATH_FROM_TEXT_PROTOCOL`.
  - [ ] chdir
  - [ ] SplitPaths
  - [ ] split_paths
  - [ ] JoinPaths
  - [ ] join_paths
  - [x] current_exe: Returns the Text representation of `EFI_DEVICE_PATH_PROTOCOL`. This can be directly converted to `EFI_DEVICE_PATH_PROTOCOL` using the `EFI_DEVICE_PATH_FROM_TEXT_PROTOCOL`.
  - [ ] Env
  - [ ] env
  - [x] *getenv: Currently using a static Guid. Maybe should use the Guid used by UefiShell for Environment Variables.
  - [x] setenv
  - [x] unsetenv
  - [ ] temp_dir
  - [ ] home_dir
  - [x] exit
  - [ ] getpid
- [x] os_str: Basically just UTF-8 strings. This is because os_str is supposed to be the superset of both the OS specific string(UCS-2) and Rust strings (UTF-8).
  - [x] Buf
  - [x] Slice
- [x] path
- [ ] pipe
  - [ ] *AnonPipe: Implemented using Runtime Variables.
    - [x] read
    - [x] read_vectored: Using default implementation
    - [x] is_read_vectored
    - [x] write
    - [x] write_vectored: Using default implementation
    - [x] is_write_vectored
    - [ ] diverge
  - [x] *read2: It is blocking and synchronous right now
- [ ] process
  - [ ] Command
    - [x] new
    - [x] arg
    - [x] env_mut
    - [ ] cwd
    - [ ] *stdin: Only null working yet.
    - [x] *stdout: Using my primitive AnonPipe
    - [x] *stderr: Using my primitive AnonPipe
    - [x] get_program
    - [x] get_args
    - [x] get_envs
    - [x] get_current_dir
    - [x] *spawn: Currently calling `EFI_BOOT_SERVICES.StartImage()` here since the pipes don't really work asynchronously right now.
  - [x] StdioPipes
  - [x] Stdio
  - [x] ExitStatus
    - [x] exit_ok
    - [x] code
  - [x] ExitStatusError
    - [x] code
  - [x] ExitCode
    - [x] as_i32
  - [ ] Process
    - [ ] id
    - [ ] kill
    - [x] wait
    - [ ] try_wait
  - [ ] CommandArgs
- [x] stdio
  - [x] *STDIN_BUF_SIZE: Currently using the same value as Windows
  - [x] Stdin
    - [x] *read: Buffered reading not implemented yet.
  - [x] Stdout
  - [x] Stderr
- [ ] thread
    - [x] sleep
- [ ] thread_local_key: Using `unsupported/thread_local_key.rs`
- [x] time
  - [x] Instant: Implemented same as `SystemTime`
  - [x] SystemTime: Implemented using `GetTime()`


All the tests for Global Allocator pass as well, so all the alloc types should work without any problem. The most fragile portions are probably `process` and `net` module. They were basically implemented for running the tests and so, only the portions that are required were implemented. Finally, the whole `libtest` mostly works, although the stdio capturing with `panic-abort-tests` option is broken (the tests do output to stdio correctly).

As for the up-streaming, the requirements aren't all that strict (since its a Tier-3 target). The current implementation should technically be enough to at least open a PR. I would like to fix a few things before opening the PR (and add the documentation of what Protocols the different portions of std use). I have been in touch with the UEFI Rust maintainer @David Rheinsberg, so it should be straightforward to get the review process going.

The PR getting merged isn't really a GSoC target (although it would certainly be great). However, getting the project to a state, where the major hurdles for merging and usability are overcome is certainly a goal. Also, I am planning on completing the merge (and possibly helping future UEFI contributions upstream), even after GSoC duration is over.


Ayush Singh


On 8/1/22 23:37, Yao, Jiewen wrote:

Hi Ayush

Thanks for the great work.

 

Would you please help me understand that how far we are between current [1] and final upstream to Rust?

 

Is upstreaming a goal of GSoC project?

 

Thank you

Yao Jiewen

 

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ayush Singh
Sent: Tuesday, August 2, 2022 1:50 AM
To: Pedro Falcato <pedro.falcato@gmail.com>; edk2-devel-groups-io <devel@edk2.groups.io>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki <mikuback@linux.microsoft.com>; Yao, Jiewen <jiewen.yao@intel.com>; Gaibusab, Jabeena B <jabeena.b.gaibusab@intel.com>
Subject: Re: [edk2-devel] Proposal to move Rust std work to a Repository under Tianocore

 

Hi Pedro. To clarify the original email. The proposal is not to create a new repository to start Rust std work. Rather, it is to move all the per-existing work that I have done for implementing std since the beginning of GSoC. This work can be found in my personal fork [1]. A significant portion of std is already in a working state for DXE UEFI and is at a point that a PR can be opened in a few weeks upstream to get it merged. A fork under Tianocore would allow more people, form both Rust and Tianocore side to experiment/improve the std, with the final goal of getting it all merged in upstream Rust.

 

Yours Sincerely

Ayush Singh

 

[1]: https://github.com/Ayush1325/rust/tree/uefi-std-rebase

 

On 8/1/22 22:56, Pedro Falcato wrote:

Hi,

 

May I suggest you just port the bare rust language (no crates, no std) to EDK2? It seems far more plausible to expect people to use a cut down version with some bindings to the rest of the project instead of hoping people just use the whole of rust, a lot of which isnt proven (or even used AFAIK) in bare metal projects. Porting just the bare minimum is way more realistic in my opinion.

 

Thanks,

Pedro

 

On Mon, 1 Aug 2022, 18:02 Ayush Singh, <ayushdevel1325@gmail.com> wrote:

Hello everyone. In the previous email thread [1], I discussed the
proposal to move Rust std work to edk2-staging and mentioned its
potential problems. After some discussion with mentors, we arrived at
the conclusion to have a rustlang [2] fork under the Tianocore
organization, and move all the std related work there. We can then open
a PR upstream from there, while allowing PRs in this repository. This
should help provide an easier and streamlined way for people to
experiment and work on this project while it is in the process of being
merged upstream.


For a status update about tests:

- passed: 12797

- failed: 40

- ignored: 375


Yours Sincerely,

Ayush Singh


[1]: https://edk2.groups.io/g/devel/message/91989

[2]: https://github.com/rust-lang/rust





--------------IBlh3E3LrjzzbhTgFfXuKwKJ--