求助python的MemoryError问题在一个项目中,我使用了python日志中的RotatingFileHandler,但是这个有一个bug
求助python的MemoryError问题
在一个项目中,我使用了python日志中的RotatingFileHandler,但是这个有一个bug,所以我自定义了一个类继承,并修改了其中的方法,不过现在有一个这样的问题,当程序连续运行2,3天,就会报如下的错误:
- Python code
Traceback (most recent call last): File "logging\__init__.pyc", line 776, in emit File "logging\__init__.pyc", line 654, in format File "logging\__init__.pyc", line 438, in format File "logging\__init__.pyc", line 404, in formatTime MemoryError
其中我写的自定义日志类如下:
- Python code
import codecsimport loggingfrom logging.handlers import RotatingFileHandlerimport osclass customLog(RotatingFileHandler): def __init__(self, filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=0): if maxBytes > 0: mode = 'a' # doesn't make sense otherwise! self.maxBytes = maxBytes self.backupCount = backupCount RotatingFileHandler.__init__(self, filename, mode, maxBytes, backupCount, encoding,delay) def doRollover(self): """ Do a rollover, as described in __init__(). """ self.stream.close() try: if self.backupCount > 0: for i in range(self.backupCount - 1, 0, -1): sfn = "%s.%d" % (self.baseFilename, i) dfn = "%s.%d" % (self.baseFilename, i + 1) if os.path.exists(sfn): # print "%s -> %s" % (sfn, dfn) if os.path.exists(dfn): os.remove(dfn) os.rename(sfn, dfn) dfn = self.baseFilename + ".1" if os.path.exists(dfn): os.remove(dfn) os.rename(self.baseFilename, dfn) # print "%s -> %s" % (self.baseFilename, dfn) except Exception: pass #print "doRollover" finally: if self.encoding: self.stream = codecs.open(self.baseFilename, 'w', self.encoding) else: self.stream = open(self.baseFilename, 'w') def shouldRollover(self, record): """ Determine if rollover should occur. Basically, see if the supplied record would cause the file to exceed the size limit we have. """ if self.stream is None: # delay was set... self.stream = self._open() if self.maxBytes > 0: # are we rolling over? msg = "" try: msg = "%s\n" % self.format(record) self.stream.seek(0, 2) #due to non-posix-compliant Windows feature except : print "shouldRollover" if self.stream.tell() + len(msg) >= self.maxBytes: return 1 return 0 def emit(self, record): """ Emit a record. Output the record to the file, catering for rollover as described in doRollover(). """ try: if self.shouldRollover(record): self.doRollover() logging.FileHandler.emit(self, record) except (KeyboardInterrupt, SystemExit): print "error" #raise except: self.handleError(record)
请问,这个问题是什么问题造成的。
[解决办法]
应该是内存泄露。下面两个链接讲了他们怎么用pdb检测/解决内存泄露的。
http://www.lshift.net/blog/2008/11/14/tracing-python-memory-leaks
http://mg.pov.lt/blog/hunting-python-memleaks
上面文章提到的方法我也很感兴趣,如果你能把你的问题用一个完整的小脚本来重现,请发出来。
