public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Bob Feng" <bob.c.feng@intel.com>
To: devel@edk2.groups.io
Cc: Liming Gao <liming.gao@intel.com>, Bob Feng <bob.c.feng@intel.com>
Subject: [Patch 09/10 V7] BaseTools: Add the support for python 2
Date: Tue,  6 Aug 2019 15:15:10 +0800	[thread overview]
Message-ID: <20190806071511.11836-10-bob.c.feng@intel.com> (raw)
In-Reply-To: <20190806071511.11836-1-bob.c.feng@intel.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

python3 change the module name of Queue to queue.
python3 add a new log handler of QueueHandler.

This patch is to make Multiple process AutoGen
feature work for python2

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
---
 .../Source/Python/AutoGen/AutoGenWorker.py    |  5 +-
 BaseTools/Source/Python/Common/EdkLogger.py   | 92 ++++++++++++++++++-
 2 files changed, 92 insertions(+), 5 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
index 1296604f688d..0a3c1d8e0ebd 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
@@ -14,11 +14,14 @@ import Common.GlobalData as GlobalData
 import Common.EdkLogger as EdkLogger
 import os
 from Common.MultipleWorkspace import MultipleWorkspace as mws
 from AutoGen.AutoGen import AutoGen
 from Workspace.WorkspaceDatabase import BuildDB
-from queue import Empty
+try:
+    from queue import Empty
+except:
+    from Queue import Empty
 import traceback
 import sys
 from AutoGen.DataPipe import MemoryDataPipe
 def clearQ(q):
     try:
diff --git a/BaseTools/Source/Python/Common/EdkLogger.py b/BaseTools/Source/Python/Common/EdkLogger.py
index f6a5e3b4daf9..15fd1458a95a 100644
--- a/BaseTools/Source/Python/Common/EdkLogger.py
+++ b/BaseTools/Source/Python/Common/EdkLogger.py
@@ -3,16 +3,100 @@
 #
 # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 
+# Copyright 2001-2016 by Vinay Sajip. All Rights Reserved.
+#
+# Permission to use, copy, modify, and distribute this software and its
+# documentation for any purpose and without fee is hereby granted,
+# provided that the above copyright notice appear in all copies and that
+# both that copyright notice and this permission notice appear in
+# supporting documentation, and that the name of Vinay Sajip
+# not be used in advertising or publicity pertaining to distribution
+# of the software without specific, written prior permission.
+# VINAY SAJIP DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING
+# ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# VINAY SAJIP BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR
+# ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
+# IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
+# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+# This copyright is for QueueHandler.
+
 ## Import modules
 from __future__ import absolute_import
 import Common.LongFilePathOs as os, sys, logging
 import traceback
 from  .BuildToolError import *
-import logging.handlers
+try:
+    from logging.handlers import QueueHandler
+except:
+    class QueueHandler(logging.Handler):
+        """
+        This handler sends events to a queue. Typically, it would be used together
+        with a multiprocessing Queue to centralise logging to file in one process
+        (in a multi-process application), so as to avoid file write contention
+        between processes.
+
+        This code is new in Python 3.2, but this class can be copy pasted into
+        user code for use with earlier Python versions.
+        """
+
+        def __init__(self, queue):
+            """
+            Initialise an instance, using the passed queue.
+            """
+            logging.Handler.__init__(self)
+            self.queue = queue
+
+        def enqueue(self, record):
+            """
+            Enqueue a record.
+
+            The base implementation uses put_nowait. You may want to override
+            this method if you want to use blocking, timeouts or custom queue
+            implementations.
+            """
+            self.queue.put_nowait(record)
+
+        def prepare(self, record):
+            """
+            Prepares a record for queuing. The object returned by this method is
+            enqueued.
+
+            The base implementation formats the record to merge the message
+            and arguments, and removes unpickleable items from the record
+            in-place.
+
+            You might want to override this method if you want to convert
+            the record to a dict or JSON string, or send a modified copy
+            of the record while leaving the original intact.
+            """
+            # The format operation gets traceback text into record.exc_text
+            # (if there's exception data), and also returns the formatted
+            # message. We can then use this to replace the original
+            # msg + args, as these might be unpickleable. We also zap the
+            # exc_info and exc_text attributes, as they are no longer
+            # needed and, if not None, will typically not be pickleable.
+            msg = self.format(record)
+            record.message = msg
+            record.msg = msg
+            record.args = None
+            record.exc_info = None
+            record.exc_text = None
+            return record
+
+        def emit(self, record):
+            """
+            Emit a record.
+
+            Writes the LogRecord to the queue, preparing it for pickling first.
+            """
+            try:
+                self.enqueue(self.prepare(record))
+            except Exception:
+                self.handleError(record)
 
 ## Log level constants
 DEBUG_0 = 1
 DEBUG_1 = 2
 DEBUG_2 = 3
@@ -206,23 +290,23 @@ def LogClientInitialize(log_q):
     # Since we use different format to log different levels of message into different
     # place (stdout or stderr), we have to use different "Logger" objects to do this.
     #
     # For DEBUG level (All DEBUG_0~9 are applicable)
     _DebugLogger.setLevel(INFO)
-    _DebugChannel = logging.handlers.QueueHandler(log_q)
+    _DebugChannel = QueueHandler(log_q)
     _DebugChannel.setFormatter(_DebugFormatter)
     _DebugLogger.addHandler(_DebugChannel)
 
     # For VERBOSE, INFO, WARN level
     _InfoLogger.setLevel(INFO)
-    _InfoChannel = logging.handlers.QueueHandler(log_q)
+    _InfoChannel = QueueHandler(log_q)
     _InfoChannel.setFormatter(_InfoFormatter)
     _InfoLogger.addHandler(_InfoChannel)
 
     # For ERROR level
     _ErrorLogger.setLevel(INFO)
-    _ErrorCh = logging.handlers.QueueHandler(log_q)
+    _ErrorCh = QueueHandler(log_q)
     _ErrorCh.setFormatter(_ErrorFormatter)
     _ErrorLogger.addHandler(_ErrorCh)
 
 ## Set log level
 #
-- 
2.20.1.windows.1


  parent reply	other threads:[~2019-08-06  7:15 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-06  7:15 [Patch 00/10 V7] Enable multiple process AutoGen Bob Feng
2019-08-06  7:15 ` [Patch 01/10 V7] BaseTools: Singleton the object to handle build conf file Bob Feng
2019-08-06  7:15 ` [Patch 02/10 V7] BaseTools: Split WorkspaceAutoGen._InitWorker into multiple functions Bob Feng
2019-08-06  7:15 ` [Patch 03/10 V7] BaseTools: Add functions to get platform scope build options Bob Feng
2019-08-06  7:15 ` [Patch 04/10 V7] BaseTools: Decouple AutoGen Objects Bob Feng
2019-08-06  7:15 ` [Patch 05/10 V7] BaseTools: Enable Multiple Process AutoGen Bob Feng
2019-08-06  7:15 ` [Patch 06/10 V7] BaseTools: Add shared data for processes Bob Feng
2019-08-06  7:15 ` [Patch 07/10 V7] BaseTools: Add LogAgent to support multiple process Autogen Bob Feng
2019-08-06  7:15 ` [Patch 08/10 V7] BaseTools: Move BuildOption parser out of build.py Bob Feng
2019-08-06  7:15 ` Bob Feng [this message]
2019-08-06  7:15 ` [Patch 10/10 V7] BaseTools: Enable block queue log agent Bob Feng

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190806071511.11836-10-bob.c.feng@intel.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox