https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/Android 与 Web 逆向工程实战手册Hugo 0.147.7 & FixIt v0.4.0-alpha.3-20251224074056-4fec1cdazh-CNoverkazaf@gmail.com (+5)overkazaf@gmail.com (+5)Tue, 16 Sep 2025 00:00:00 +0000https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/hbase/Tue, 16 Sep 2025 00:00:00 +0000overkazaf@gmail.com (+5)https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/hbase/<h1 class="heading-element" id="hbase-分布式-nosql-数据库"><span>HBase 分布式 NoSQL 数据库</span> <a href="#hbase-%e5%88%86%e5%b8%83%e5%bc%8f-nosql-%e6%95%b0%e6%8d%ae%e5%ba%93" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h1><p>Apache HBase 是一个分布式、面向列的开源数据库，基于 Google Bigtable 论文实现，运行在 HDFS 之上。</p>https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/data_warehousing_and_processing/Sun, 17 Aug 2025 00:00:00 +0000overkazaf@gmail.com (+5)https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/data_warehousing_and_processing/<h1 class="heading-element" id="数据仓库与计算引擎"><span>数据仓库与计算引擎</span> <a href="#%e6%95%b0%e6%8d%ae%e4%bb%93%e5%ba%93%e4%b8%8e%e8%ae%a1%e7%ae%97%e5%bc%95%e6%93%8e" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h1><h2 class="heading-element" id="当通过逆向和爬虫采集到海量数据后例如数亿条用户行为日志商品信息如何存储管理和分析这些数据就成了大数据领域的核心问题本节将介绍主流的数据仓库和分布式计算引擎技术"><span>当通过逆向和爬虫采集到海量数据后（例如，数亿条用户行为日志、商品信息），如何存储、管理和分析这些数据，就成了大数据领域的核心问题。本节将介绍主流的数据仓库和分布式计算引擎技术。</span> <a href="#%e5%bd%93%e9%80%9a%e8%bf%87%e9%80%86%e5%90%91%e5%92%8c%e7%88%ac%e8%99%ab%e9%87%87%e9%9b%86%e5%88%b0%e6%b5%b7%e9%87%8f%e6%95%b0%e6%8d%ae%e5%90%8e%e4%be%8b%e5%a6%82%e6%95%b0%e4%ba%bf%e6%9d%a1%e7%94%a8%e6%88%b7%e8%a1%8c%e4%b8%ba%e6%97%a5%e5%bf%97%e5%95%86%e5%93%81%e4%bf%a1%e6%81%af%e5%a6%82%e4%bd%95%e5%ad%98%e5%82%a8%e7%ae%a1%e7%90%86%e5%92%8c%e5%88%86%e6%9e%90%e8%bf%99%e4%ba%9b%e6%95%b0%e6%8d%ae%e5%b0%b1%e6%88%90%e4%ba%86%e5%a4%a7%e6%95%b0%e6%8d%ae%e9%a2%86%e5%9f%9f%e7%9a%84%e6%a0%b8%e5%bf%83%e9%97%ae%e9%a2%98%e6%9c%ac%e8%8a%82%e5%b0%86%e4%bb%8b%e7%bb%8d%e4%b8%bb%e6%b5%81%e7%9a%84%e6%95%b0%e6%8d%ae%e4%bb%93%e5%ba%93%e5%92%8c%e5%88%86%e5%b8%83%e5%bc%8f%e8%ae%a1%e7%ae%97%e5%bc%95%e6%93%8e%e6%8a%80%e6%9c%af" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><h2 class="heading-element" id="1-数据仓库-data-warehouse"><span>1. 数据仓库 (Data Warehouse)</span> <a href="#1-%e6%95%b0%e6%8d%ae%e4%bb%93%e5%ba%93-data-warehouse" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>数据仓库是一个用于存储和分析海量结构化、半结构化数据的系统。它与业务数据库（OLTP）不同，其核心目标是支持复杂的分析查询（OLAP）。</p>https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/flink/Sat, 09 Aug 2025 00:00:00 +0000overkazaf@gmail.com (+5)https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/flink/<h1 class="heading-element" id="apache-flink-实时流处理"><span>Apache Flink 实时流处理</span> <a href="#apache-flink-%e5%ae%9e%e6%97%b6%e6%b5%81%e5%a4%84%e7%90%86" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h1><p>Apache Flink 是一个分布式流处理框架，专为低延迟、高吞吐量的实时数据处理而设计。</p>https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/spark/Tue, 10 Jun 2025 00:00:00 +0000overkazaf@gmail.com (+5)https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/spark/<h1 class="heading-element" id="apache-spark-大数据处理"><span>Apache Spark 大数据处理</span> <a href="#apache-spark-%e5%a4%a7%e6%95%b0%e6%8d%ae%e5%a4%84%e7%90%86" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h1><p>Apache Spark 是一个统一的大数据处理引擎，支持批处理、流处理、机器学习和图计算。</p>https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/hive/Fri, 25 Apr 2025 00:00:00 +0000overkazaf@gmail.com (+5)https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/hive/<h1 class="heading-element" id="hive-数据仓库"><span>Hive 数据仓库</span> <a href="#hive-%e6%95%b0%e6%8d%ae%e4%bb%93%e5%ba%93" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h1><p>Apache Hive 是基于 Hadoop 的数据仓库工具，可以将结构化的数据文件映射为数据库表，并提供类 SQL 查询功能。</p>https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/message_queues/Sat, 09 Aug 2025 00:00:00 +0000overkazaf@gmail.com (+5)https://overkazaf.github.io/reverse_engineering/android/04-reference/engineering/data-analysis/message_queues/<h1 class="heading-element" id="消息队列"><span>消息队列</span> <a href="#%e6%b6%88%e6%81%af%e9%98%9f%e5%88%97" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h1><p>消息队列（Message Queue）是分布式系统中实现异步通信、解耦组件的核心中间件，在逆向工程数据管线中扮演着至关重要的角色。</p>