From: "Bob Feng" <bob.c.feng@intel.com>
To: devel@edk2.groups.io
Cc: Liming Gao <liming.gao@intel.com>, Bob Feng <bob.c.feng@intel.com>
Subject: [Patch 09/10 V5] BaseTools: Add the support for python 2
Date: Wed, 31 Jul 2019 13:52:43 +0800 [thread overview]
Message-ID: <20190731055244.19872-10-bob.c.feng@intel.com> (raw)
In-Reply-To: <20190731055244.19872-1-bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
python3 change the module name of Queue to queue.
python3 add a new log handler of QueueHandler.
This patch is to make Multiple process AutoGen
feature work for python2
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
---
.../Source/Python/AutoGen/AutoGenWorker.py | 5 +-
BaseTools/Source/Python/Common/EdkLogger.py | 92 ++++++++++++++++++-
2 files changed, 92 insertions(+), 5 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
index 12637fac963d..1f388ee178be 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
@@ -14,11 +14,14 @@ import Common.GlobalData as GlobalData
import Common.EdkLogger as EdkLogger
import os
from Common.MultipleWorkspace import MultipleWorkspace as mws
from AutoGen.AutoGen import AutoGen
from Workspace.WorkspaceDatabase import BuildDB
-from queue import Empty
+try:
+ from queue import Empty
+except:
+ from Queue import Empty
import traceback
import sys
from AutoGen.DataPipe import MemoryDataPipe
def clearQ(q):
try:
diff --git a/BaseTools/Source/Python/Common/EdkLogger.py b/BaseTools/Source/Python/Common/EdkLogger.py
index f6a5e3b4daf9..15fd1458a95a 100644
--- a/BaseTools/Source/Python/Common/EdkLogger.py
+++ b/BaseTools/Source/Python/Common/EdkLogger.py
@@ -3,16 +3,100 @@
#
# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
+# Copyright 2001-2016 by Vinay Sajip. All Rights Reserved.
+#
+# Permission to use, copy, modify, and distribute this software and its
+# documentation for any purpose and without fee is hereby granted,
+# provided that the above copyright notice appear in all copies and that
+# both that copyright notice and this permission notice appear in
+# supporting documentation, and that the name of Vinay Sajip
+# not be used in advertising or publicity pertaining to distribution
+# of the software without specific, written prior permission.
+# VINAY SAJIP DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING
+# ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# VINAY SAJIP BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR
+# ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
+# IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
+# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+# This copyright is for QueueHandler.
+
## Import modules
from __future__ import absolute_import
import Common.LongFilePathOs as os, sys, logging
import traceback
from .BuildToolError import *
-import logging.handlers
+try:
+ from logging.handlers import QueueHandler
+except:
+ class QueueHandler(logging.Handler):
+ """
+ This handler sends events to a queue. Typically, it would be used together
+ with a multiprocessing Queue to centralise logging to file in one process
+ (in a multi-process application), so as to avoid file write contention
+ between processes.
+
+ This code is new in Python 3.2, but this class can be copy pasted into
+ user code for use with earlier Python versions.
+ """
+
+ def __init__(self, queue):
+ """
+ Initialise an instance, using the passed queue.
+ """
+ logging.Handler.__init__(self)
+ self.queue = queue
+
+ def enqueue(self, record):
+ """
+ Enqueue a record.
+
+ The base implementation uses put_nowait. You may want to override
+ this method if you want to use blocking, timeouts or custom queue
+ implementations.
+ """
+ self.queue.put_nowait(record)
+
+ def prepare(self, record):
+ """
+ Prepares a record for queuing. The object returned by this method is
+ enqueued.
+
+ The base implementation formats the record to merge the message
+ and arguments, and removes unpickleable items from the record
+ in-place.
+
+ You might want to override this method if you want to convert
+ the record to a dict or JSON string, or send a modified copy
+ of the record while leaving the original intact.
+ """
+ # The format operation gets traceback text into record.exc_text
+ # (if there's exception data), and also returns the formatted
+ # message. We can then use this to replace the original
+ # msg + args, as these might be unpickleable. We also zap the
+ # exc_info and exc_text attributes, as they are no longer
+ # needed and, if not None, will typically not be pickleable.
+ msg = self.format(record)
+ record.message = msg
+ record.msg = msg
+ record.args = None
+ record.exc_info = None
+ record.exc_text = None
+ return record
+
+ def emit(self, record):
+ """
+ Emit a record.
+
+ Writes the LogRecord to the queue, preparing it for pickling first.
+ """
+ try:
+ self.enqueue(self.prepare(record))
+ except Exception:
+ self.handleError(record)
## Log level constants
DEBUG_0 = 1
DEBUG_1 = 2
DEBUG_2 = 3
@@ -206,23 +290,23 @@ def LogClientInitialize(log_q):
# Since we use different format to log different levels of message into different
# place (stdout or stderr), we have to use different "Logger" objects to do this.
#
# For DEBUG level (All DEBUG_0~9 are applicable)
_DebugLogger.setLevel(INFO)
- _DebugChannel = logging.handlers.QueueHandler(log_q)
+ _DebugChannel = QueueHandler(log_q)
_DebugChannel.setFormatter(_DebugFormatter)
_DebugLogger.addHandler(_DebugChannel)
# For VERBOSE, INFO, WARN level
_InfoLogger.setLevel(INFO)
- _InfoChannel = logging.handlers.QueueHandler(log_q)
+ _InfoChannel = QueueHandler(log_q)
_InfoChannel.setFormatter(_InfoFormatter)
_InfoLogger.addHandler(_InfoChannel)
# For ERROR level
_ErrorLogger.setLevel(INFO)
- _ErrorCh = logging.handlers.QueueHandler(log_q)
+ _ErrorCh = QueueHandler(log_q)
_ErrorCh.setFormatter(_ErrorFormatter)
_ErrorLogger.addHandler(_ErrorCh)
## Set log level
#
--
2.20.1.windows.1
next prev parent reply other threads:[~2019-07-31 5:53 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-31 5:52 [Patch 00/10 V5] Enable multiple process AutoGen Bob Feng
2019-07-31 5:52 ` [Patch 01/10 V5] BaseTools: Singleton the object to handle build conf file Bob Feng
2019-07-31 5:52 ` [Patch 02/10 V5] BaseTools: Split WorkspaceAutoGen._InitWorker into multiple functions Bob Feng
2019-07-31 11:33 ` [edk2-devel] " Philippe Mathieu-Daudé
2019-07-31 12:46 ` Bob Feng
2019-07-31 5:52 ` [Patch 03/10 V5] BaseTools: Add functions to get platform scope build options Bob Feng
2019-07-31 5:52 ` [Patch 04/10 V5] BaseTools: Decouple AutoGen Objects Bob Feng
2019-07-31 5:52 ` [Patch 05/10 V5] BaseTools: Enable Multiple Process AutoGen Bob Feng
2019-07-31 5:52 ` [Patch 06/10 V5] BaseTools: Add shared data for processes Bob Feng
2019-07-31 5:52 ` [Patch 07/10 V5] BaseTools: Add LogAgent to support multiple process Autogen Bob Feng
2019-07-31 5:52 ` [Patch 08/10 V5] BaseTools: Move BuildOption parser out of build.py Bob Feng
2019-07-31 5:52 ` Bob Feng [this message]
2019-07-31 5:52 ` [Patch 10/10 V5] BaseTools: Enable block queue log agent Bob Feng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190731055244.19872-10-bob.c.feng@intel.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox