python 实现过滤出tomcat日志中含有ERROR 或Exception 的行并保存在另一个文件

遍历多个tomcat日志文件,找出含有ERROR 和Exception 的日志,并把该行日志输出到另一个文件中:(这里为了体现python模块导入的知识,所有建立了多个文件夹和模块)

项目结构:

consetting.py:
# 日志文件目录
F_PATH = r'C:UsersshenpingPycharmProjectsShenping_TESTday_5scriptglivelogs'
# 错误日志存储目录
D_PATH = r'C:UsersshenpingPycharmProjectsShenping_TESTday_5scriptglivedata'




rwfile.py:
def op_file(filename,content =None):
    f = open(filename,'a+',encoding='utf-8')
    f.seek(0)
    if content:
        f.writelines(content)
        res = None
    else:
        res = f.readlines()
    f.close()
    return res



operationlog.py:
import re,os,datetime
from lib.rwfile import op_file

# 遍历日志文件:
# 找出含有 error 或 exception 的行,按格式写入文件
def operation_log(filename,f_path,d_path):
    index = 0
    log_file = os.path.join(f_path, filename)
    f = op_file(log_file)
    lis = []  # 存储错误日志
    for j in f:
        index += 1
        m = re.search('error',j,re.IGNORECASE)
        n = re.search('exception',j,re.IGNORECASE)
        if m or n:
            data_file = os.path.join(d_path, str(datetime.date.today())+'_tomcat.log')
            log_str = "文件名:"+filename+""+str(index)+"行:"+j+'
'
            lis.append(log_str)
    op_file(data_file,lis)
 

 start.py:

# 启动文件
import os,sys

cur_path = os.path.abspath(__file__)
base_dir = os.path.dirname(os.path.dirname(cur_path))
sys.path.insert(0,base_dir)
from conf.consetting import F_PATH,D_PATH
from lib.operationlog import operation_log

log_name = os.listdir(F_PATH)
for x in log_name:
    operation_log(x,F_PATH,D_PATH)
    print('========================================华丽的分割线=========================================================')

运行后结果:

文件名:catalina.2017-10-21.log 第5行:21-Oct-2017 14:16:32.118 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-21.log 第9行:21-Oct-2017 14:16:32.119 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [threadDeathWatcher-2-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-21.log 第14行:21-Oct-2017 14:16:32.120 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [logback-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-21.log 第515行:21-Oct-2017 16:08:39.217 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [glive-task-pool-1-thread-12] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-22.log 第216行:22-Oct-2017 15:17:49.169 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-22.log 第248行:22-Oct-2017 15:17:54.089 Exception [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property.
文件名:catalina.2017-10-22.log 第249行:22-Oct-2017 15:17:54.113 Exception [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Valve} Setting property 'resolveHosts' to 'false' did not find a matching property.
文件名:catalina.2017-10-23.log 第3行:23-Oct-2017 10:51:55.461 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
文件名:catalina.2017-10-23.log 第4行:23-Oct-2017 10:51:55.462 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.fabric.jdbc.FabricMySQLDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
文件名:catalina.2017-10-23.log 第9行:23-Oct-2017 10:51:55.464 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [threadDeathWatcher-3-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-23.log 第3356行:23-Oct-2017 21:47:50.795 IllegalStateException [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3357行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3398行:23-Oct-2017 21:47:50.800 ERROR [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3399行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3444行:23-Oct-2017 21:47:50.802 ERROR [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3445行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3491行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3532行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.RollingPolicyBaseCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3570行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.TimeBasedRollingPolicyCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3605行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.helper.DateTokenConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3637行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.INFOStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-23.log 第3667行: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.INFOStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名:catalina.2017-10-24.log 第5行:24-Oct-2017 08:02:51.428 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-24.log 第520行:24-Oct-2017 14:42:09.350 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名:catalina.2017-10-24.log 第670行:24-Oct-2017 14:42:09.374 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@e5d976d]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@b0bd2a4]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
文件名:catalina.2017-10-24.log 第682行:24-Oct-2017 14:42:09.376 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55c7591]) and a value of type [io.grpc.Context] (value [io.grpc.Context@390c5a57]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
文件名:catalina.2017-10-24.log 第704行:24-Oct-2017 14:42:09.378 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55c7591]) and a value of type [io.grpc.Context] (value [io.grpc.Context@390c5a57]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
原文地址:https://www.cnblogs.com/wolfshining/p/7725571.html