PHP项目SRE与可用性保障

wen PHP项目 1

PHP项目SRE与可用性保障

核心监控体系

应用层监控

PHP项目SRE与可用性保障

// 自定义监控埋点示例
class AppMonitor {
    public static function trackRequest($startTime, $status) {
        $duration = microtime(true) - $startTime;
        // 记录到StatsD/InfluxDB
        StatsD::timing('app.request.duration', $duration);
        StatsD::increment("app.request.status.{$status}");
        // 慢请求告警
        if ($duration > 2.0) {
            Alert::warning("Slow request detected", [
                'duration' => $duration,
                'uri' => $_SERVER['REQUEST_URI']
            ]);
        }
    }
}

基础设施监控

  • CPU/Memory:使用Prometheus + Node Exporter
  • PHP-FPM状态/status 页面监控
  • MySQL:慢查询、连接数、复制延迟
  • Redis:命中率、内存使用、阻塞操作

高可用架构设计

负载均衡层

用户请求 → CDN → Nginx(负载均衡) → PHP-FPM集群
                                    → PHP-FPM集群

会话管理

// Redis会话存储配置
'session' => [
    'handler' => 'redis',
    'path' => 'tcp://redis-sentinel:26379?timeout=2',
    'ttl' => 7200,
    'options' => [
        'prefix' => 'sess_',
    ]
]

数据库高可用

# 数据库主从架构
master:
  - host: 10.0.1.1
  - port: 3306
slaves:
  - host: 10.0.1.2
    weight: 10
  - host: 10.0.1.3
    weight: 5
# 读写分离配置
'database' => [
    'read' => ['10.0.1.2', '10.0.1.3'],
    'write' => ['10.0.1.1']
]

故障自动恢复

健康检查脚本

#!/bin/bash
# health_check.sh
check_phpfpm() {
    local pool=$1
    local status=$(curl -s http://127.0.0.1/status | grep "idle processes")
    if [ -z "$status" ]; then
        echo "PHP-FPM pool $pool unhealthy"
        return 1
    fi
    return 0
}
auto_recover() {
    local service=$1
    case $service in
        "php-fpm")
            systemctl restart php7.4-fpm
            sleep 5
            # 检查恢复情况
            check_phpfpm && return 0
            ;;
        "nginx")
            nginx -t && systemctl reload nginx
            ;;
    esac
    return 1
}
# 主循环
while true; do
    if ! check_phpfpm; then
        auto_recover "php-fpm"
        send_alert "PHP-FPM auto-recovered"
    fi
    sleep 30
done

自动扩缩容策略

# Docker Compose自动扩缩容
services:
  php-app:
    image: php-app:latest
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '0.5'
          memory: 256M
      restart_policy:
        condition: any
        delay: 5s
      update_config:
        parallelism: 2
        delay: 10s

性能优化策略

PHP-FPM调优

; php-fpm.conf
pm = dynamic
pm.max_children = 50
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 1000
request_terminate_timeout = 30s
; 进程管理优化
listen.backlog = 65535
pm.status_path = /status
ping.path = /ping

OPcache配置

[opcache]
opcache.enable=1
opcache.memory_consumption=256
opcache.interned_strings_buffer=16
opcache.max_accelerated_files=10000
opcache.revalidate_freq=2
opcache.fast_shutdown=1
opcache.validate_timestamps=0  ; 生产环境关闭

Nginx静态资源缓存

location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
    expires 30d;
    add_header Cache-Control "public, no-transform";
    add_header Vary "Accept-Encoding";
    # 版本化资源缓存
    if ($query_string ~* "v=\d+") {
        expires max;
    }
}

日志与追踪

集中式日志管理

# Filebeat配置
filebeat.inputs:
- type: log
  paths:
    - /var/log/php-fpm/*.log
    - /var/log/nginx/*.log
output.elasticsearch:
  hosts: ["elasticsearch:9200"]
  index: "php-logs-%{+yyyy.MM.dd}"
# 日志格式结构化
logstash:
  filter:
    - grok:
        match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:message}" }
    - date:
        match => ["timestamp", "ISO8601"]

分布式追踪

// OpenTracing集成
use OpenTracing\GlobalTracer;
class TraceMiddleware {
    public function handle($request, Closure $next) {
        $span = GlobalTracer::get()->startSpan('request');
        $span->setTag('http.method', $request->method());
        $span->setTag('http.url', $request->fullUrl());
        try {
            $response = $next($request);
            $span->setTag('http.status_code', $response->status());
            return $response;
        } catch (\Exception $e) {
            $span->setTag('error', true);
            $span->log(['message' => $e->getMessage()]);
            throw $e;
        } finally {
            $span->finish();
        }
    }
}

灾难恢复计划

备份策略

# 数据库备份
mysqldump --single-transaction --quick --lock-tables=false \
    --databases app_db | gzip > /backup/db_$(date +%Y%m%d).sql.gz
# 文件备份
rsync -avz --delete /var/www/html/ backup@10.0.1.100:/backup/
# 配置备份
etcdctl backup /backup/etcd_$(date +%Y%m%d)

恢复流程

#!/bin/bash
# disaster_recovery.sh
# 1. 检查服务状态
check_services() {
    local services=("nginx" "php7.4-fpm" "mysql" "redis")
    for service in "${services[@]}"; do
        if ! systemctl is-active --quiet $service; then
            systemctl restart $service
        fi
    done
}
# 2. 恢复数据库
restore_database() {
    local backup_file=$1
    local db_name=$2
    mysql -e "CREATE DATABASE IF NOT EXISTS $db_name"
    gunzip < $backup_file | mysql $db_name
}
# 3. 验证数据完整性
verify_data() {
    # 检查关键表
    local tables=("users" "orders" "products")
    for table in "${tables[@]}"; do
        local count=$(mysql -N -e "SELECT COUNT(*) FROM $table")
        if [ $count -eq 0 ]; then
            echo "WARNING: $table is empty"
            return 1
        fi
    done
}
# 主恢复流程
main() {
    check_services
    restore_database "/backup/db_20231201.sql.gz" "app_db"
    verify_data
    # 启动应用
    systemctl restart php7.4-fpm
    systemctl reload nginx
    # 检查状态
    curl -f http://localhost/health || {
        echo "Recovery failed"
        exit 1
    }
}

SRE关键指标

SLA/SLO定义

  • 可用性:99.95%(每年宕机不超过4.38小时)
  • 响应时间:P95 < 500ms,P99 < 2s
  • 错误率:< 0.1%
  • 吞吐量:每秒处理请求数 > 1000

告警分级

  • P0:服务完全不可用 → 立即响应(5分钟内)
  • P1:核心功能受损 → 15分钟响应
  • P2:非核心功能异常 → 1小时内处理
  • P3:性能下降 → 24小时内优化

持续优化

容量规划

# 基于历史数据的容量预测
def capacity_planning(historical_data, growth_rate=0.3):
    current_capacity = historical_data['peak_rps'][-1]
    future_peak = current_capacity * (1 + growth_rate) ** 12
    required_servers = math.ceil(future_peak / SERVER_CAPACITY)
    return {
        'future_peak_rps': future_peak,
        'recommended_servers': required_servers,
        'deployment_plan': f'Add {required_servers} servers in next quarter'
    }

混沌工程

# 定期执行混沌测试
chaos_experiments = [
    {
        'name': 'kill_php_fpm_random',
        'target': 'php-fpm',
        'action': 'kill_random_process',
        'frequency': 'daily',
        'duration': 60,
        'expected_behavior': 'auto_restart'
    },
    {
        'name': 'network_latency',
        'target': 'database',
        'action': 'add_100ms_latency',
        'frequency': 'weekly',
        'duration': 300,
        'expected_behavior': 'connection_pool_timeout'
    }
]

通过以上SRE实践,PHP项目可以实现99.95%以上的可用性,同时具备快速故障恢复和自动扩缩容能力,持续的性能优化和灾备演练是保障系统长期稳定运行的关键。

抱歉,评论功能暂时关闭!