PHP项目SRE与可用性保障
核心监控体系
应用层监控

// 自定义监控埋点示例
class AppMonitor {
public static function trackRequest($startTime, $status) {
$duration = microtime(true) - $startTime;
// 记录到StatsD/InfluxDB
StatsD::timing('app.request.duration', $duration);
StatsD::increment("app.request.status.{$status}");
// 慢请求告警
if ($duration > 2.0) {
Alert::warning("Slow request detected", [
'duration' => $duration,
'uri' => $_SERVER['REQUEST_URI']
]);
}
}
}
基础设施监控
- CPU/Memory:使用Prometheus + Node Exporter
- PHP-FPM状态:
/status页面监控 - MySQL:慢查询、连接数、复制延迟
- Redis:命中率、内存使用、阻塞操作
高可用架构设计
负载均衡层
用户请求 → CDN → Nginx(负载均衡) → PHP-FPM集群
→ PHP-FPM集群
会话管理
// Redis会话存储配置
'session' => [
'handler' => 'redis',
'path' => 'tcp://redis-sentinel:26379?timeout=2',
'ttl' => 7200,
'options' => [
'prefix' => 'sess_',
]
]
数据库高可用
# 数据库主从架构
master:
- host: 10.0.1.1
- port: 3306
slaves:
- host: 10.0.1.2
weight: 10
- host: 10.0.1.3
weight: 5
# 读写分离配置
'database' => [
'read' => ['10.0.1.2', '10.0.1.3'],
'write' => ['10.0.1.1']
]
故障自动恢复
健康检查脚本
#!/bin/bash
# health_check.sh
check_phpfpm() {
local pool=$1
local status=$(curl -s http://127.0.0.1/status | grep "idle processes")
if [ -z "$status" ]; then
echo "PHP-FPM pool $pool unhealthy"
return 1
fi
return 0
}
auto_recover() {
local service=$1
case $service in
"php-fpm")
systemctl restart php7.4-fpm
sleep 5
# 检查恢复情况
check_phpfpm && return 0
;;
"nginx")
nginx -t && systemctl reload nginx
;;
esac
return 1
}
# 主循环
while true; do
if ! check_phpfpm; then
auto_recover "php-fpm"
send_alert "PHP-FPM auto-recovered"
fi
sleep 30
done
自动扩缩容策略
# Docker Compose自动扩缩容
services:
php-app:
image: php-app:latest
deploy:
replicas: 3
resources:
limits:
cpus: '0.5'
memory: 256M
restart_policy:
condition: any
delay: 5s
update_config:
parallelism: 2
delay: 10s
性能优化策略
PHP-FPM调优
; php-fpm.conf pm = dynamic pm.max_children = 50 pm.start_servers = 10 pm.min_spare_servers = 5 pm.max_spare_servers = 20 pm.max_requests = 1000 request_terminate_timeout = 30s ; 进程管理优化 listen.backlog = 65535 pm.status_path = /status ping.path = /ping
OPcache配置
[opcache] opcache.enable=1 opcache.memory_consumption=256 opcache.interned_strings_buffer=16 opcache.max_accelerated_files=10000 opcache.revalidate_freq=2 opcache.fast_shutdown=1 opcache.validate_timestamps=0 ; 生产环境关闭
Nginx静态资源缓存
location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
expires 30d;
add_header Cache-Control "public, no-transform";
add_header Vary "Accept-Encoding";
# 版本化资源缓存
if ($query_string ~* "v=\d+") {
expires max;
}
}
日志与追踪
集中式日志管理
# Filebeat配置
filebeat.inputs:
- type: log
paths:
- /var/log/php-fpm/*.log
- /var/log/nginx/*.log
output.elasticsearch:
hosts: ["elasticsearch:9200"]
index: "php-logs-%{+yyyy.MM.dd}"
# 日志格式结构化
logstash:
filter:
- grok:
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:message}" }
- date:
match => ["timestamp", "ISO8601"]
分布式追踪
// OpenTracing集成
use OpenTracing\GlobalTracer;
class TraceMiddleware {
public function handle($request, Closure $next) {
$span = GlobalTracer::get()->startSpan('request');
$span->setTag('http.method', $request->method());
$span->setTag('http.url', $request->fullUrl());
try {
$response = $next($request);
$span->setTag('http.status_code', $response->status());
return $response;
} catch (\Exception $e) {
$span->setTag('error', true);
$span->log(['message' => $e->getMessage()]);
throw $e;
} finally {
$span->finish();
}
}
}
灾难恢复计划
备份策略
# 数据库备份
mysqldump --single-transaction --quick --lock-tables=false \
--databases app_db | gzip > /backup/db_$(date +%Y%m%d).sql.gz
# 文件备份
rsync -avz --delete /var/www/html/ backup@10.0.1.100:/backup/
# 配置备份
etcdctl backup /backup/etcd_$(date +%Y%m%d)
恢复流程
#!/bin/bash
# disaster_recovery.sh
# 1. 检查服务状态
check_services() {
local services=("nginx" "php7.4-fpm" "mysql" "redis")
for service in "${services[@]}"; do
if ! systemctl is-active --quiet $service; then
systemctl restart $service
fi
done
}
# 2. 恢复数据库
restore_database() {
local backup_file=$1
local db_name=$2
mysql -e "CREATE DATABASE IF NOT EXISTS $db_name"
gunzip < $backup_file | mysql $db_name
}
# 3. 验证数据完整性
verify_data() {
# 检查关键表
local tables=("users" "orders" "products")
for table in "${tables[@]}"; do
local count=$(mysql -N -e "SELECT COUNT(*) FROM $table")
if [ $count -eq 0 ]; then
echo "WARNING: $table is empty"
return 1
fi
done
}
# 主恢复流程
main() {
check_services
restore_database "/backup/db_20231201.sql.gz" "app_db"
verify_data
# 启动应用
systemctl restart php7.4-fpm
systemctl reload nginx
# 检查状态
curl -f http://localhost/health || {
echo "Recovery failed"
exit 1
}
}
SRE关键指标
SLA/SLO定义
- 可用性:99.95%(每年宕机不超过4.38小时)
- 响应时间:P95 < 500ms,P99 < 2s
- 错误率:< 0.1%
- 吞吐量:每秒处理请求数 > 1000
告警分级
- P0:服务完全不可用 → 立即响应(5分钟内)
- P1:核心功能受损 → 15分钟响应
- P2:非核心功能异常 → 1小时内处理
- P3:性能下降 → 24小时内优化
持续优化
容量规划
# 基于历史数据的容量预测
def capacity_planning(historical_data, growth_rate=0.3):
current_capacity = historical_data['peak_rps'][-1]
future_peak = current_capacity * (1 + growth_rate) ** 12
required_servers = math.ceil(future_peak / SERVER_CAPACITY)
return {
'future_peak_rps': future_peak,
'recommended_servers': required_servers,
'deployment_plan': f'Add {required_servers} servers in next quarter'
}
混沌工程
# 定期执行混沌测试
chaos_experiments = [
{
'name': 'kill_php_fpm_random',
'target': 'php-fpm',
'action': 'kill_random_process',
'frequency': 'daily',
'duration': 60,
'expected_behavior': 'auto_restart'
},
{
'name': 'network_latency',
'target': 'database',
'action': 'add_100ms_latency',
'frequency': 'weekly',
'duration': 300,
'expected_behavior': 'connection_pool_timeout'
}
]
通过以上SRE实践,PHP项目可以实现99.95%以上的可用性,同时具备快速故障恢复和自动扩缩容能力,持续的性能优化和灾备演练是保障系统长期稳定运行的关键。