java案例如何利用历史大数据建模预测？

wen java案例 2026-11-15 1

Java案例：利用历史大数据建模预测

下面给出一个完整可运行的Java案例，涵盖从历史数据加载、特征工程、模型训练、预测到评估的全流程，以销售预测为例（可根据业务替换为销量、流量、库存、故障等）。

整体架构

历史数据(CSV/DB) 
   → 数据清洗与特征工程 
   → 模型训练(线性回归/时间序列/ML库) 
   → 模型持久化 
   → 在线预测 
   → 评估与迭代

Java生态常用方案：

场景	推荐库
传统机器学习	Smile、Weka、Apache Spark MLlib
深度学习	DJL (Deep Java Library)、ONNX Runtime
大数据量	Spark MLlib + Java API
轻量线性/统计	自己实现或 Apache Commons Math

依赖（Maven）

<dependency>
    <groupId>com.github.haifengl</groupId>
    <artifactId>smile-core</artifactId>
    <version>3.0.2</version>
</dependency>
<dependency>
    <groupId>com.github.haifengl</groupId>
    <artifactId>smile-data</artifactId>
    <version>3.0.2</version>
</dependency>

案例：基于历史销量预测未来销量

数据样本（sales_history.csv）

date,dayOfWeek,isHoliday,promotion,price,lastWeekSales,sales
2024-01-01,1,1,0,19.9,120,150
2024-01-02,2,0,0,19.9,150,142
...

字段说明：

dayOfWeek 星期几
isHoliday 是否节假日
promotion 是否促销
price 单价
lastWeekSales 上周同期销量（滞后特征）
sales 目标值

实体类

public class SalesRecord {
    public double dayOfWeek;
    public double isHoliday;
    public double promotion;
    public double price;
    public double lastWeekSales;
    public double sales; // 标签
}

数据加载与特征工程

import java.io.*;
import java.nio.file.*;
import java.util.*;
public class DataLoader {
    public static double[][] loadFeatures(String path) throws IOException {
        List<String> lines = Files.readAllLines(Paths.get(path));
        double[][] data = new double[lines.size() - 1][6];
        for (int i = 1; i < lines.size(); i++) {
            String[] cols = lines.get(i).split(",");
            // 跳过 date 列(0)，取 dayOfWeek 起
            for (int j = 0; j < 6; j++) {
                data[i - 1][j] = Double.parseDouble(cols[j + 1]);
            }
        }
        return data;
    }
    public static double[] extractLabels(double[][] data) {
        double[] y = new double[data.length];
        for (int i = 0; i < data.length; i++) y[i] = data[i][5];
        return y;
    }
    public static double[][] extractFeatures(double[][] data) {
        double[][] X = new double[data.length][5];
        for (int i = 0; i < data.length; i++)
            System.arraycopy(data[i], 0, X[i], 0, 5);
        return X;
    }
}

训练模型（Smile 线性回归 + 交叉验证）

import smile.data.formula.Formula;
import smile.data.type.*;
import smile.data.vector.*;
import smile.data.DataFrame;
import smile.regression.*;
import smile.validation.*;
public class TrainModel {
    public static void main(String[] args) throws Exception {
        double[][] data = DataLoader.loadFeatures("data/sales_history.csv");
        double[][] X = DataLoader.extractFeatures(data);
        double[] y = DataLoader.extractLabels(data);
        // 构造 Smile DataFrame
        StructType schema = DataTypes.struct(
            new StructField("dayOfWeek", DataTypes.DoubleType),
            new StructField("isHoliday", DataTypes.DoubleType),
            new StructField("promotion", DataTypes.DoubleType),
            new StructField("price",     DataTypes.DoubleType),
            new StructField("lastWeekSales", DataTypes.DoubleType),
            new StructField("sales",     DataTypes.DoubleType)
        );
        double[][] full = new double[data.length][6];
        for (int i = 0; i < data.length; i++) {
            System.arraycopy(X[i], 0, full[i], 0, 5);
            full[i][5] = y[i];
        }
        DataFrame df = DataFrame.of(full, schema);
        // 训练线性回归
        Formula formula = Formula.lhs("sales");
        LinearRegression model = LinearRegression.fit(formula, df);
        System.out.println("模型系数: " + model.formula());
        System.out.println("截距: " + model.intercept());
        // 预测一条
        double[] sample = new double[]{3, 0, 1, 18.5, 140, 0}; // 最后一位占位
        double pred = model.predict(sample);
        System.out.println("预测销量: " + pred);
        // 5 折交叉验证评估
        double[] preds = CrossValidation.regression(5, df,
                (train, test) -> LinearRegression.fit(formula, train),
                (m, test) -> test.stream()
                        .mapToDouble(r -> m.predict(r))
                        .toArray()
        );
        System.out.println("RMSE: " + RMSE.of(y, preds));
        System.out.println("R²:   " + R2.of(y, preds));
        // 模型持久化
        java.io.ObjectOutputStream out = new java.io.ObjectOutputStream(
                new java.io.FileOutputStream("model/lr.model"));
        out.writeObject(model);
        out.close();
        System.out.println("模型已保存");
    }
}

在线预测服务

import java.io.*;
public class PredictService {
    private LinearRegression model;
    public PredictService(String modelPath) throws Exception {
        try (ObjectInputStream in = new ObjectInputStream(
                new FileInputStream(modelPath))) {
            model = (LinearRegression) in.readObject();
        }
    }
    /** 输入：dayOfWeek, isHoliday, promotion, price, lastWeekSales */
    public double predict(double dayOfWeek, double isHoliday,
                          double promotion, double price, double lastWeek) {
        double[] x = new double[]{dayOfWeek, isHoliday, promotion, price, lastWeek, 0};
        return model.predict(x);
    }
    public static void main(String[] args) throws Exception {
        PredictService svc = new PredictService("model/lr.model");
        double p = svc.predict(5, 0, 1, 18.9, 130);
        System.out.printf("明天预计销量: %.2f 件%n", p);
    }
}

进阶：使用时间序列或 Spark 大数据建模

若数据量大（TB级）→ Spark MLlib

import org.apache.spark.sql.*;
import org.apache.spark.ml.regression.LinearRegression;
import org.apache.spark.ml.feature.VectorAssembler;
SparkSession spark = SparkSession.builder()
    .appName("SalesForecast").master("local[*]").getOrCreate();
Dataset<Row> df = spark.read().option("header", true)
    .option("inferSchema", true).csv("hdfs:///data/sales/*.csv");
VectorAssembler assembler = new VectorAssembler()
    .setInputCols(new String[]{"dayOfWeek","isHoliday","promotion","price","lastWeekSales"})
    .setOutputCol("features");
Dataset<Row> train = assembler.transform(df).select("features","sales");
LinearRegression lr = new LinearRegression()
    .setLabelCol("sales").setFeaturesCol("features")
    .setMaxIter(100).setRegParam(0.1);
var model = lr.fit(train);
model.save("hdfs:///model/sales_lr");

若强调时间序列（季节性）→ 使用 Smile 的 ARIMA/时序

import smile.timeseries.ARIMA;
double[] series = /* 历史销量序列 */;
ARIMA arima = ARIMA.fit(series, 1, 1, 1); // p,d,q
double[] forecast = arima.forecast(7);    // 预测未来7天

若需要深度学习 → DJL

// 加载 ONNX 训练好的模型或用 DJL 训练 MLP/ LSTM
Criteria<NDList, NDList> criteria = Criteria.builder()
    .setTypes(NDList.class, NDList.class)
    .optModelPath(Paths.get("model/lstm.zip"))
    .optEngine("PyTorch")
    .build();
try (ZooModel<NDList, NDList> m = criteria.loadModel();
     Predictor<NDList, NDList> p = m.newPredictor()) {
    // 构造输入 NDArray, 得到预测
}

工程落地要点

环节	建议
数据质量	缺失值填充、异常值剔除、去重
特征工程	滞后特征(lag)、滑动均值、季节因子、节假日独热编码
模型选择	简单场景线性回归；非线性和交互用 GBDT(RandomForest/GradientBoost)；时序用 ARIMA/Prophet(可调用Python)；深度学习用 DJL
离线训练 + 在线预测	训练结果序列化，服务中只加载预测，避免每次重训
模型版本管理	保存模型版本号 + 训练数据快照，便于回滚
监控	记录预测值 vs 实际值，计算漂移，触发重训
A/B 与灰度	新模型先小流量验证再全量