Sentinel控制台监控数据持久化到MySQL数据库

Posted on by hackdl

阅读文本大概需要3分钟。

       根据官方wiki文档，Sentinel控制台的实时监控数据，默认仅存储 5 分钟以内的数据。如需持久化，需要定制实现相关接口。

https://github.com/alibaba/Sentinel/wiki/在生产环境中使用-Sentinel

给出了指导步骤：

  • 自行扩展实现 MetricsRepository 接口；
  • 注册成 Spring Bean 并在相应位置通过 @Qualifier 注解指定对应的 bean name 即可；

0x01：MetricsRepository接口定义

package com.alibaba.csp.sentinel.dashboard.repository.metric;

import java.util.List;


public interface MetricsRepository {

    void save(T metric);

    void saveAll(Iterable metrics);

    List queryByAppAndResourceBetween(String app, String resource, long startTime, long endTime);

    List listResourcesOfApp(String app);
}

该接口就只定义4个方法，分别用于保存和查询Sentinel的metric数据。注释其实很清楚了，解析如下：

  • save：保存单个metric
  • saveAll：保存多个metric
  • queryByAppAndResourceBetween：通过应用名称、资源名称、开始时间、结束时间查询metric列表
  • listResourcesOfApp：通过应用名称查询资源列表

目前该接口只有一个基于内存级别的实现类：com.alibaba.csp.sentinel.dashboard.repository.metric.InMemoryMetricsRepository。

另外还有一个实体类com.alibaba.csp.sentinel.dashboard.datasource.entity.MetricEntity，如下图

梳理了相关的类关系就可以实现了。

0x02：根据MetricEntity新建数据库和新建实体类

建表语句如下

-- 创建监控数据表
CREATE TABLE `t_sentinel_metric` (
  `id` INT NOT NULL AUTO_INCREMENT COMMENT 'id，主键',
  `gmt_create` DATETIME COMMENT '创建时间',
  `gmt_modified` DATETIME COMMENT '修改时间',
  `app` VARCHAR(100) COMMENT '应用名称',
  `timestamp` DATETIME COMMENT '统计时间',
  `resource` VARCHAR(500) COMMENT '资源名称',
  `pass_qps` INT COMMENT '通过qps',
  `success_qps` INT COMMENT '成功qps',
  `block_qps` INT COMMENT '限流qps',
  `exception_qps` INT COMMENT '发送异常的次数',
  `rt` DOUBLE COMMENT '所有successQps的rt的和',
  `_count` INT COMMENT '本次聚合的总条数',
  `resource_code` INT COMMENT '资源的hashCode',
  INDEX app_idx(`app`) USING BTREE,
  INDEX resource_idx(`resource`) USING BTREE,
  INDEX timestamp_idx(`timestamp`) USING BTREE,
  PRIMARY KEY (`id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8;

实体类如下

package com.alibaba.csp.sentinel.dashboard.datasource.entity;

import javax.persistence.*;
import java.io.Serializable;
import java.util.Date;

/**
 * @author 2230
 *
 */
@Entity
@Table(name = "t_sentinel_metric")
public class MetricDto implements Serializable {

    private static final long serialVersionUID = 7200023615444172715L;

    /**id，主键*/
    @Id
    @GeneratedValue
    @Column(name = "id")
    private Long id;

    /**创建时间*/
    @Column(name = "gmt_create")
    private Date gmtCreate;

    /**修改时间*/
    @Column(name = "gmt_modified")
    private Date gmtModified;

    /**应用名称*/
    @Column(name = "app")
    private String app;

    /**统计时间*/
    @Column(name = "timestamp")
    private Date timestamp;

    /**资源名称*/
    @Column(name = "resource")
    private String resource;

    /**通过qps*/
    @Column(name = "pass_qps")
    private Long passQps;

    /**成功qps*/
    @Column(name = "success_qps")
    private Long successQps;

    /**限流qps*/
    @Column(name = "block_qps")
    private Long blockQps;

    /**发送异常的次数*/
    @Column(name = "exception_qps")
    private Long exceptionQps;

    /**所有successQps的rt的和*/
    @Column(name = "rt")
    private Double rt;

    /**本次聚合的总条数*/
    @Column(name = "_count")
    private Integer count;

    /**资源的hashCode*/
    @Column(name = "resource_code")
    private Integer resourceCode;

   // get  set 方法省略

}

0x03：pom.xml添加依赖

因为是基于JPA和MySQL数据库实现，所以需要添加JPA依赖和MySQL数据库驱动依赖


          org.springframework.boot
          spring-boot-starter-data-jpa
          ${spring.boot.version}
        

        
            mysql
            mysql-connector-java
            5.1.47

0x04：实现MetricsRepository 接口，把数据持久化到MySQL数据库

注意实现添加@Repository(“jpaMetricsRepository”)配置

package com.alibaba.csp.sentinel.dashboard.repository.metric;

import java.time.Instant;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.Query;

import org.springframework.beans.BeanUtils;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.util.CollectionUtils;

import com.alibaba.csp.sentinel.dashboard.datasource.entity.MetricDto;
import com.alibaba.csp.sentinel.dashboard.datasource.entity.MetricEntity;
import com.alibaba.csp.sentinel.util.StringUtil;


 * https://github.com/alibaba/spring-cloud-alibaba/wiki/Sentinel
 * 
 * @author 2230
 *
 */
@Transactional
@Repository("jpaMetricsRepository")
public class JpaMetricsRepository implements MetricsRepository {

    @PersistenceContext
    private EntityManager em;

    @Override
    public void save(MetricEntity metric) {
        if (metric == null || StringUtil.isBlank(metric.getApp())) {
            return;
        }

        MetricDto metricDto = new MetricDto();
        BeanUtils.copyProperties(metric, metricDto);
        em.persist(metricDto);
    }

    @Override
    public void saveAll(Iterable metrics) {
        if (metrics == null) {
            return;
        }

        metrics.forEach(this::save);
    }

    @Override
    public List queryByAppAndResourceBetween(String app, String resource, long startTime, long endTime) {
        List results = new ArrayList();
        if (StringUtil.isBlank(app)) {
            return results;
        }

        if (StringUtil.isBlank(resource)) {
            return results;
        }

        StringBuilder hql = new StringBuilder();
        hql.append("FROM MetricDto");
        hql.append(" WHERE app=:app");
        hql.append(" AND resource=:resource");
        hql.append(" AND timestamp>=:startTime");
        hql.append(" AND timestamp metricDtos = query.getResultList();
        if (CollectionUtils.isEmpty(metricDtos)) {
            return results;
        }

        for (MetricDto metricDto : metricDtos) {
            MetricEntity metricEntity = new MetricEntity();
            BeanUtils.copyProperties(metricDto, metricEntity);
            results.add(metricEntity);
        }
        return results;
    }

    @Override
    public List listResourcesOfApp(String app) {
        List results = new ArrayList();
        if (StringUtil.isBlank(app)) {
            return results;
        }

        StringBuilder hql = new StringBuilder();
        hql.append("FROM MetricDto");
        hql.append(" WHERE app=:app");
        hql.append(" AND timestamp>=:startTime");

        long startTime = System.currentTimeMillis() - 1000 * 60;
        Query query = em.createQuery(hql.toString());
        query.setParameter("app", app);
        query.setParameter("startTime", Date.from(Instant.ofEpochMilli(startTime)));

        List metricDtos = query.getResultList();
        if (CollectionUtils.isEmpty(metricDtos)) {
            return results;
        }

        List metricEntities = new ArrayList();
        for (MetricDto metricDto : metricDtos) {
            MetricEntity metricEntity = new MetricEntity();
            BeanUtils.copyProperties(metricDto, metricEntity);
            metricEntities.add(metricEntity);
        }

        Map resourceCount = new HashMap(32);

        for (MetricEntity metricEntity : metricEntities) {
            String resource = metricEntity.getResource();
            if (resourceCount.containsKey(resource)) {
                MetricEntity oldEntity = resourceCount.get(resource);
                oldEntity.addPassQps(metricEntity.getPassQps());
                oldEntity.addRtAndSuccessQps(metricEntity.getRt(), metricEntity.getSuccessQps());
                oldEntity.addBlockQps(metricEntity.getBlockQps());
                oldEntity.addExceptionQps(metricEntity.getExceptionQps());
                oldEntity.addCount(1);
            } else {
                resourceCount.put(resource, MetricEntity.copyOf(metricEntity));
            }
        }

        // Order by last minute b_qps DESC.
        return resourceCount.entrySet()
                .stream()
                .sorted((o1, o2) -> {
                    MetricEntity e1 = o1.getValue();
                    MetricEntity e2 = o2.getValue();
                    int t = e2.getBlockQps().compareTo(e1.getBlockQps());
                    if (t != 0) {
                        return t;
                    }
                    return e2.getPassQps().compareTo(e1.getPassQps());
                })
                .map(Map.Entry::getKey)
                .collect(Collectors.toList());
    }
}

0x05：application.properties配置文件添加数据库配置

# datasource
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/gateway_v2?characterEncoding=utf8&useSSL=true
spring.datasource.username=root
spring.datasource.password=root

# spring data jpa
spring.jpa.hibernate.ddl-auto=none
spring.jpa.hibernate.use-new-id-generator-mappings=false
spring.jpa.database-platform=org.hibernate.dialect.MySQLDialect
spring.jpa.show-sql=false

主要配置数据库连接信息和JPA的配置项，JPA使用Hibernate实现。

0x06：数据库持久化换成JpaMetricsRepository实现

找到如下两个类

com.alibaba.csp.sentinel.dashboard.controller.MetricController
com.alibaba.csp.sentinel.dashboard.metric.MetricFetcher

在metricStore属性上添加多一个@Qualifier(“jpaMetricsRepository”)注解，如下图

0x07：验证

设置sentinel-dashboard工程的启动参数

-Dserver.port=8080 -Dcsp.sentinel.dashboard.server=localhost:8080 -Dproject.name=sentinel-dashboard

具体可以参考【 Sentinel如何进行流量监控 】；可以发现数据已经保存到MySQL数据库。

备注：以上代码改造都是在sentinel-dashboard项目上。

关注我

每天进步一点点

服务器托管，北京服务器托管，服务器租用 http://www.fwqtg.net
机房租用，北京机房租用，IDC机房托管， http://www.fwqtg.net

相关推荐: 什么是好代码/坏代码？给普通人的图解示例

本文翻译自国外论坛 medium，原文地址：https://medium.com/@todbotts.triangles/what-is-good-bad-code-an-illustrated-example-for-non-programmers-1222…

服务器托管，北京服务器托管，服务器租用，机房机柜带宽租用

服务器托管

咨询：董先生

电话13051898268 QQ/微信93663045！

上一篇:
下一篇: