NginxЧÀÍÆ÷µÄÈÕÖ¾ÆÊÎöºÍ¼à¿ØÊÖ¶ÎÏêϸÏÈÈÝ
NginxЧÀÍÆ÷µÄÈÕÖ¾ÆÊÎöºÍ¼à¿ØÊÖ¶ÎÏêϸÏÈÈÝ
¸ÅÊö£º
NginxÊÇÒ»¸ö¸ßÐÔÄܵÄWebЧÀÍÆ÷ºÍ·´ÏòÊðÀíЧÀÍÆ÷£¬ÆÕ±éÓ¦ÓÃÓÚÖÖÖÖ»¥ÁªÍøÓ¦Óó¡¾°¡£ÔÚÏÖʵӦÓÃÖУ¬ÎÒÃǾ³£ÐèÒª¶ÔNginxЧÀÍÆ÷µÄÈÕÖ¾¾ÙÐÐÆÊÎöºÍ¼à¿Ø£¬ÒÔ±ã¾ÙÐйÊÕÏÅŲ顢ÐÔÄÜÓÅ»¯ºÍÇå¾²·À»¤¡£±¾ÎĽ«ÏêϸÏÈÈÝÔõÑùͨ¹ýÖÖÖÖÊֶζÔNginxЧÀÍÆ÷µÄÈÕÖ¾¾ÙÐÐÆÊÎöºÍ¼à¿Ø¡£
Ò»¡¢ÉèÖÃNginxÈÕÖ¾ÃûÌÃ
NginxµÄÈÕÖ¾ÊäÌØÊâʽ¿ÉÒÔͨ¹ýÉèÖÃÎļþ¾ÙÐÐÉèÖá£ÔÚNginxµÄÉèÖÃÎļþÖУ¬¿ÉÒÔͨ¹ýÔÚhttp»òserver¿éÖÐʹÓÃaccess_logÖ¸ÁîÀ´½ç˵ÈÕÖ¾ÃûÌúÍÊä³öÄ¿µÄ¡£ÏÂÃæÊÇÒ»¸ö¼òÆÓµÄʾÀý£º
http { log_format mylog '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"'; access_log /var/log/nginx/access.log mylog; }
µÇ¼ºó¸´ÖÆ
ÔÚÉÏÊöʾÀýÖУ¬ÎÒÃǽç˵ÁËÒ»¸öÃûΪmylogµÄÈÕÖ¾ÃûÌ㬲¢½«ÈÕÖ¾Êä³öµ½Îļþ/var/log/nginx/access.logÖС£¸ÃÈÕÖ¾ÃûÌðüÀ¨ÁËNginxЧÀÍÆ÷ÎüÊÕµ½µÄÿ¸öÇëÇóµÄÏà¹ØÐÅÏ¢¡£
¶þ¡¢Ê¹ÓÃÏÂÁîÐÐÆÊÎöNginxÈÕÖ¾
NginxµÄÈÕÖ¾ÎļþÒ»Ñùƽ³£ÊÇÒÔ´¿Îı¾µÄÐÎʽÉúÑÄÔÚЧÀÍÆ÷ÉÏ¡£ÎÒÃÇ¿ÉÒÔʹÓÃÏÂÁîÐй¤¾ß¶ÔNginxÈÕÖ¾¾ÙÐÐÆÊÎöºÍͳ¼Æ¡£ÏÂÃæÊÇһЩ³£ÓõÄÏÂÁîÐй¤¾ß£º
grep£ºÓÃÓÚÔÚÎı¾ÖÐËÑË÷Ö¸¶¨µÄ×Ö·û´®£¬¿ÉÒÔÓÃÀ´¹ýÂ˳öÖª×ãÌض¨Ìõ¼þµÄÈÕÖ¾¼Í¼¡£
# ¹ýÂ˳ö°üÀ¨Òªº¦×Ö¡°404¡±µÄÈÕÖ¾¼Í¼ $ grep "404" /var/log/nginx/access.log
µÇ¼ºó¸´ÖÆ
awk£ºÓÃÓÚ¶ÔÎı¾¾ÙÐÐÖ§½â¡¢¹ýÂ˺ʹ¦Öóͷ££¬¿ÉÒÔÓÃÀ´ÌáÈ¡ÈÕÖ¾¼Í¼ÖеÄÌض¨×ֶΡ£
# ÌáÈ¡³ö»á¼ûIPºÍÏìӦ״̬ÂëµÄ×ֶΠ$ awk '{print $1" "$9}' /var/log/nginx/access.log
µÇ¼ºó¸´ÖÆ
sed£ºÓÃÓÚ¶ÔÎı¾¾ÙÐÐÌæ»»¡¢É¾³ýºÍ²åÈë²Ù×÷£¬¿ÉÒÔÓÃÀ´ÐÞËûÈÕÖ¾¼Í¼µÄÃûÌá£
# ½«ÈÕÖ¾ÖеÄIPµØµãÌ滻Ϊ¡°x.x.x.x¡± $ sed 's/[0-9]+.[0-9]+.[0-9]+.[0-9]+/x.x.x.x/' /var/log/nginx/access.log
µÇ¼ºó¸´ÖÆ
Èý¡¢Ê¹ÓÃELK Stack¾ÙÐÐNginxÈÕÖ¾ÆÊÎöºÍ¼à¿Ø
ELK StackÊÇÒ»Ì׿ªÔ´µÄÈÕÖ¾ÖÎÀí¹¤¾ß£¬ÓÉElasticsearch¡¢LogstashºÍKibana×é³É¡£ÏÂÃ潫ÏÈÈÝÔõÑùʹÓÃELK Stack¶ÔNginxÈÕÖ¾¾ÙÐÐÆÊÎöºÍ¼à¿Ø¡£
×°ÖúÍÉèÖÃElasticsearchºÍKibana£º
Ê×ÏÈ£¬ÐèҪװÖúÍÉèÖÃElasticsearchºÍKibanaЧÀÍ¡£ÕâЩ°ì·¨¿ÉÒÔÔÚ¹Ù·½ÎĵµÖÐÕÒµ½ÏêϸµÄ˵Ã÷¡£
ÉèÖÃLogstash£º
LogstashÊÇÒ»¸öÓÃÓÚÈÕÖ¾ÍøÂç¡¢´¦Öóͷ£ºÍת·¢µÄ¹¤¾ß¡£ÎÒÃÇÐèÒªÉèÖÃLogstashÒÔÎüÊÕNginxÈÕÖ¾£¬²¢½«Æä·¢Ë͵½Elasticsearch¾ÙÐд洢ºÍË÷Òý¡£ÏÂÃæÊÇÒ»¸ö¼òÆÓµÄLogstashÉèÖÃʾÀý£º
input { file { path => "/var/log/nginx/access.log" start_position => "beginning" } } filter { grok { match => { "message" => "%{IPORHOST:clientip} - %{DATA:user_ident} [%{HTTPDATE:timestamp}] "%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:http_version}" %{NUMBER:status} %{NUMBER:bytes_sent} "%{DATA:http_referer}" "%{DATA:http_user_agent}"" } } } output { elasticsearch { hosts => ["localhost:9200"] index => "nginx-access-%{+YYYY.MM.dd}" } }
µÇ¼ºó¸´ÖÆ
ÔÚÉÏÊöʾÀýÖУ¬ÎÒÃÇʹÓÃÁËgrok²å¼þÀ´ÆÊÎöNginxÈÕÖ¾¼Í¼¡£Logstash½«ÆÊÎöºóµÄ×ֶη¢Ë͵½Elasticsearch¾ÙÐÐË÷Òý£¬²¢°´ÈÕÆÚ»®·Ö´æ´¢¡£
ʹÓÃKibana¾ÙÐÐÈÕÖ¾ÆÊÎöºÍ¼à¿Ø£º
Æô¶¯LogstashЧÀͺó£¬ÎÒÃÇ¿ÉÒÔͨ¹ýKibana½çÃæ¾ÙÐÐÈÕÖ¾ÆÊÎöºÍ¼à¿Ø¡£ÔÚKibanaÖУ¬ÎÒÃÇ¿ÉÒÔ½¨ÉèÒDZíÅÌ¡¢Í¼±íºÍ¾¯±¨À´Õ¹Ê¾ºÍ¼à¿ØNginxÈÕÖ¾µÄÏà¹ØÖ¸±ê¡£ÔÚKibana¿ØÖÆ̨ÖУ¬ÎÒÃÇ¿ÉÒÔʹÓÃElasticsearchÅÌÎÊÓïÑÔ£¨ÈçLuceneºÍKQL£©¾ÙÐÐÊý¾ÝɸѡºÍ¾ÛºÏ£¬ÒÔ±ã¿ìËÙÕÒµ½ËùÐèµÄÐÅÏ¢¡£
½áÓ
NginxЧÀÍÆ÷µÄÈÕÖ¾ÆÊÎöºÍ¼à¿ØÊÇÔËάÊÂÇéÖÐÖ÷ÒªµÄÒ»²¿·Ö¡£Í¨¹ýÉÏÊöÏÈÈݵÄÒªÁ죬ÎÒÃÇ¿ÉÒÔÎÞаµØ¶ÔNginxÈÕÖ¾¾ÙÐÐÆÊÎöºÍ¼à¿Ø£¬´Ó¶øʵʱ·¢Ã÷ÎÊÌâºÍ¾ÙÐÐÐÔÄÜÓÅ»¯¡£ÎÞÂÛÊÇʹÓÃÏÂÁîÐй¤¾ßÕÕ¾ÉELK Stack¹¤¾ß£¬Ö»ÒªÕÆÎÕÁËÏìÓ¦µÄ¼¼ÇɺÍÒªÁ죬ÎÒÃǾÍÄܹ»¸üºÃµØÖÎÀíºÍά»¤NginxЧÀÍÆ÷¡£
ÒÔÉϾÍÊÇNginxЧÀÍÆ÷µÄÈÕÖ¾ÆÊÎöºÍ¼à¿ØÊÖ¶ÎÏêϸÏÈÈݵÄÏêϸÄÚÈÝ£¬¸ü¶àÇë¹Ø×¢±¾ÍøÄÚÆäËüÏà¹ØÎÄÕ£¡