误差序列分析
<p>[TOC]</p>
<p>本模块的函数族旨在提供对数据误差分组综合检验，同时提供误差频率分布，平均误差、平均绝对误差、均方根误差、准确率等检验指标的综合展示。在浏览本模块函数说明之前可先参阅函数设计的<a href="https://www.showdoc.cc/meteva?page_id=4072178802924517">关键技术思想</a></p>
<pre><code class="language-python">%matplotlib inline
%load_ext autoreload
%autoreload 2
import meteva.base as meb
import meteva.method as mem
import meteva.product as mpd
import numpy as np
import datetime
import copy
import matplotlib.pyplot as plt
import pandas as pd</code></pre>
<p><a href="https://www.showdoc.cc/nmc?page_id=3831227192066999">本模块测试数据集简介和更详细的数据收集代码说明</a> </p>
<pre><code class="language-python">###################以下开始为数据收集部分的程序
#设置关注的起始时段
time_start= datetime.datetime(2019,7,1,8,0)
time_end = datetime.datetime(2019,8,1,8,0)
#读取站点列表，并将站点内容为缺省值，当其作为读取站点数据的参数时，如果站点文件中某个站号不存在时,返回结果中该站点保持为缺省值
station = meb.read_stadata_from_micaps3(r"H:\test_data\input\mpd\Example_data\ob\temp_2m\BT19070102.000")
station.iloc[:,-1] = meb.IV
##读取收集观测数据
dir_ob = r"H:\test_data\input\mpd\Example_data\ob\temp_2m\BTYYMMDDHH.000" # 观测数据的路径模板
sta_list = []
time0 = time_start
while time0 < time_end:
path = meb.get_path(dir_ob,time0)
sta = meb.read_stadata_from_micaps3(path,station = station,time = time0,dtime = 0,level = 0,data_name = "ob")
sta_list.append(sta)
time0 += datetime.timedelta(hours = 3)
ob_sta_all = pd.concat(sta_list,axis = 0) #数据拼接
ob_sta_all = meb.sele_by_para(ob_sta_all,value=[-100,100]) #简单的质量控制
#读取收集ec预报数据
dir_ec = r"H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\YYMMDD\BTYYMMDDHH.TTT" #ECMWF data path patten
sta_list =[]
time0 = time_start
while time0 <= time_end:
for dh in range(0,73,3):
path = meb.get_path(dir_ec,time0,dh)
grd = meb.read_griddata_from_micaps4(path)
if grd is not None:
sta = meb.interp_gs_linear(grd,station)
meb.set_stadata_coords(sta,time = time0,dtime = dh,level = 0)
meb.set_stadata_names(sta,["ECMWF"])
sta_list.append(sta)
time0 += datetime.timedelta(hours = 12) #此处跳着读是为了减少测试用时
ec_sta_all = pd.concat(sta_list,axis = 0) #数据拼接
#读取收集grapes预报数据
dir_grapes = r"H:\test_data\input\mpd\Example_data\grapes\temp_2m\YYMMDD\BTYYMMDDHH.TTT"
sta_list =[]
time0 = time_start
while time0 <= time_end:
for dh in range(0,73,3):
path = meb.get_path(dir_grapes,time0,dh)
grd = meb.read_griddata_from_micaps4(path)
if grd is not None:
sta = meb.interp_gs_linear(grd,station)
meb.set_stadata_coords(sta,time = time0,dtime = dh,level = 0)
meb.set_stadata_names(sta,["grapes"])
sta_list.append(sta)
time0 += datetime.timedelta(hours = 12)
grapes_sta_all = pd.concat(sta_list,axis = 0) #数据拼接
#数据匹配合并
sta_all = meb.combine_on_obTime_id(ob_sta_all,[ec_sta_all,grapes_sta_all])
sta_all = meb.sele_by_para(sta_all,drop_IV = True) #删除包含缺省值的样本
###################以上为数据收集部分的程序</code></pre>
<pre><code>H:\test_data\input\mpd\Example_data\ob\temp_2m\BT19071111.000文件不存在
H:\test_data\input\mpd\Example_data\ob\temp_2m\BT19071114.000文件不存在
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.000 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.003 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.006 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.009 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.012 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.015 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.018 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.021 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.024 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.027 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.030 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.033 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.036 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.039 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.042 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.045 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.048 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.051 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.054 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.057 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.060 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.063 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.066 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.069 does not exist
H:\test_data\input\mpd\Example_data\ecmwf\temp_2m\190801\BT19080108.072 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.000 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.003 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.006 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.009 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.012 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.015 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.018 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.021 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.024 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.027 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.030 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.033 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.036 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.039 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.042 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.045 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.048 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.051 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.054 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.057 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.060 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.063 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.066 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.069 does not exist
H:\test_data\input\mpd\Example_data\grapes\temp_2m\190801\BT19080108.072 does not exist</code></pre>
<h1>误差综合分析图</h1>
<p><strong><font face="黑体" color=blue size=3>error_boxplot(sta_ob_and_fos,s = None,g = None,gll=None,group_name_list=None,threshold = 2,save_dir=None,save_path = None,show = False,title="误差综合分析图",spasify_xticks = None,sup_fontsize =10,width = None,height = None,</strong>kwargs)</font>**<br />
根据输入的站点数据，选择部分数据，将选择的数据进行进行分组检验，绘制误差综合分析图 </p>
<h1>误差综合分析图（绝对值）</h1>
<p><strong><font face="黑体" color=blue size=3>error_boxplot_abs(sta_ob_and_fos,s = None,g = None,gll=None,group_name_list=None,threshold = 2,save_dir=None,save_path = None,show = False,title="绝对误差综合分析图",spasify_xticks = None, sup_fontsize = 10, width = None, height = None,</strong>kwargs)</font>**<br />
和根据输入的站点数据，选择部分数据，将选择的数据进行进行分组检验，绘制误差综合分析图 </p>
<table>
<thead>
<tr>
<th style="text-align: left;">参数</th>
<th style="text-align: left;">说明（上述两个函数参数意义和用法完全相同）</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left;"><strong><font face="黑体" color=blue size=5>sta_ob_and_fos</font></strong></td>
<td style="text-align: left;">实况和预报合并对齐后的数据，形式为站点数据格式如上述例子中的sta_all</td>
</tr>
<tr>
<td style="text-align: left;"><strong>s</strong></td>
<td style="text-align: left;">用于选择数据样本的字典参数，具体的参数说明可参见meb.sele_by_dict中的<a href="https://www.showdoc.cc/meteva?page_id=3975604785954540"><font face="黑体" color=red size=5>s</font></a>参数</td>
</tr>
<tr>
<td style="text-align: left;"><strong>g</strong></td>
<td style="text-align: left;">用于分组检验的参数，具体用法可参见meb.group中的<a href="https://www.showdoc.cc/meteva?page_id=4071849185300418"><font face="黑体" color=red size=5>g</font></a>参数</td>
</tr>
<tr>
<td style="text-align: left;"><strong>gll</strong></td>
<td style="text-align: left;">用于分组检验的参数，具体用法可参见meb.group中的<a href="https://www.showdoc.cc/meteva?page_id=4071849185300418"><font face="黑体" color=red size=5>gll</font></a>参数</td>
</tr>
<tr>
<td style="text-align: left;"><strong>group_name_list</strong></td>
<td style="text-align: left;">指定分组的名称，它将被作为绘图的x轴坐标</td>
</tr>
<tr>
<td style="text-align: left;"><strong>threshold</strong></td>
<td style="text-align: left;">计算准确率使用的误差阈值</td>
</tr>
<tr>
<td style="text-align: left;"><strong>save_dir</strong></td>
<td style="text-align: left;">图片输出路径</td>
</tr>
<tr>
<td style="text-align: left;"><strong>save_path</strong></td>
<td style="text-align: left;">指定图片输出的文件路径，当批量生成多张图片时save_path为包含所有图片的输出路径的列表</td>
</tr>
<tr>
<td style="text-align: left;"><strong>show</strong></td>
<td style="text-align: left;">是否在屏幕显示图片，如果save_dir 为None时，程序内部会自动将show设置True</td>
</tr>
<tr>
<td style="text-align: left;"><strong>title</strong></td>
<td style="text-align: left;">图片标题的确定分为1、全自动，2、半自动，3手动三种方式。<br>1、全自动：不设置title参数，系统自动采用title的缺省值+自动补齐的其它信息来确定每一幅图的标题；<br> 2、半自动：title 为字符串类型，系统会采用title + 自动补齐的其它信息来确定每一幅图的标题；<br>3、手动：title为一个包含多个字符串的列表，且列表的长度必须和要绘制的图的数量一致，每一幅图会依次采用列表中的字符串作为标题</td>
</tr>
<tr>
<td style="text-align: left;"><strong>spasify_xticks</strong></td>
<td style="text-align: left;">x轴坐标刻度的稀疏倍数，缺省时会自动进行稀疏化,当spasify_xticks = 1 时, 所有的刻度值都要显示, spasify_xticks = 2时，则间隔1个刻度进行显示，依次类推</td>
</tr>
<tr>
<td style="text-align: left;"><strong>sup_fontsize</strong></td>
<td style="text-align: left;">图片标题的字体大小，其它字体将根据标题字体大小自动设置，其中坐标轴字体大小 = sup_fontsize <em> 0.9, 坐标刻度的字体大小 = sup_fontsize </em> 0.8</td>
</tr>
<tr>
<td style="text-align: left;"><strong>width</strong></td>
<td style="text-align: left;">图片的宽度，缺省时程序自动设置</td>
</tr>
<tr>
<td style="text-align: left;"><strong>height</strong></td>
<td style="text-align: left;">图片的高度，缺省时程序自动设置</td>
</tr>
<tr>
<td style="text-align: left;"><strong>show</strong></td>
<td style="text-align: left;">是否已plt.show()形式在屏幕显示图片</td>
</tr>
<tr>
<td style="text-align: left;">**kwargs</td>
<td style="text-align: left;">用于接受matplotlib 的boxplot函数的参数，下面提供了一个使用示例，更多参数设置请参考(<a href="https://matplotlib.org/3.3.1/api/_as_gen/matplotlib.pyplot.boxplot.html">https://matplotlib.org/3.3.1/api/_as_gen/matplotlib.pyplot.boxplot.html</a>)</td>
</tr>
<tr>
<td style="text-align: left;"><font face="黑体" color=blue size=3>return</font></td>
<td style="text-align: left;">无返回值</td>
</tr>
</tbody>
</table>
<p><strong>调用示例：</strong></p>
<pre><code class="language-python">mpd.error_boxplot(sta_all,s = {"month":[7,8],"hour":8},g = "dtime",
save_dir = r"H:\test_data\output\mpd\program",show = True
) #提取7，8月份的数据，按时效分类检验</code></pre>
<pre><code>图片已保存至H:\test_data\output\mpd\program/ECMWF.png</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/2564fef90eb9c2cccac6c0ea427b95a2" alt="" /></p>
<pre><code>图片已保存至H:\test_data\output\mpd\program/grapes.png</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/257e85b7a7ce31dafab4124d98be5f3d" alt="" /></p>
<pre><code class="language-python">#mpd.error_boxplot(sta_all,s = {"dtime":[24]},g = "month",gll=[[1,2,3],[4,5,6],[7,8,9],[10,11,12]],save_dir = r"H:\test_data\output\mpd\program")</code></pre>
<pre><code class="language-python">mpd.error_boxplot(sta_all,s = {"hour":8},g = "ob_hour")</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/18b24410ce2bc90555ce52532b7da56d" alt="" /></p>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/5a507ba2eef198fd82049cdfb408bbe2" alt="" /></p>
<pre><code class="language-python">mpd.error_boxplot(sta_all,s = {"member":["ob","ECMWF"]},g = "id")</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/99e7c3846eb21e8d97239b49d7411992" alt="" /></p>
<pre><code class="language-python">mpd.error_boxplot_abs(sta_all,g = "time") </code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/07b0ab226c4ab836fef6bc70646ccf28" alt="" /></p>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/671ce88cf2a87a43b909314b7ebb3cf5" alt="" /></p>
<pre><code class="language-python">#提取7，8月份的ecmwf数据，按时效分类检验（箱须图显示误差绝对值分布）
mpd.error_boxplot_abs(sta_all,s = {"month":[7,8],"member":["ob","ECMWF"]},g = "dtime",showfliers = False,vmax = 10) </code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/3b300a415fc0538c9d95e6db12e50fed" alt="" /></p>
<pre><code class="language-python">mpd.error_boxplot(sta_all,g = "time",title = "误差时间序列图") # 设置tiitle中相同的部分，title中模式名称仍然会自动添加</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/2cb1e15a6d5dd6e9b51c6a390eee8ddc" alt="" /></p>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/7f61a86cc8dc5eee8866992c073e6c14" alt="" /></p>
<pre><code class="language-python">mpd.error_boxplot_abs(sta_all,g = "time",
save_path=[r"H:\test_data\output\mpd\program\EC模式",r"H:\test_data\output\mpd\program\gradpes模式"],show = True,
title = ["ECMWF模式","GRAPES模式"])
#绘制的图片不为单张时，若需指定输出路径，save_path 参数需为数组形式。
#title为列表时，所有图的titleH:\test_data\output\mpd\program完全有用户指定</code></pre>
<pre><code>图片已保存至H:\test_data\output\mpd\program\EC模式</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/43cb7c381185771808fd6ad677a3b00b" alt="" /></p>
<pre><code>图片已保存至H:\test_data\output\mpd\program\gradpes模式</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/d5a9797620916e76af1a8b200573470e" alt="" /></p>
<pre><code class="language-python">mpd.error_boxplot(sta_all,g = "time",s = {"member":["ob","ECMWF"]},
json_path=r"H:\test_data\output\mpd\program\json_error_boxplot.txt")
#绘制的图片不为单张时，若需指定输出路径，save_path 参数需为数组形式。
#title为列表时，所有图的titleH:\test_data\output\mpd\program完全有用户指定</code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/53a7ea0a761fd73a222fe183dd84de47" alt="" /></p>
<pre><code>have printed pictrue elements to H:\test_data\output\mpd\program\json_error_boxplot.txt</code></pre>
<pre><code class="language-python"># **kwargs参数使用示例
# 下面的示例中通过设置showfliers = False，可将箱须图中的异常点不显示
# 其中showfliers 是boxplot的一个参数
mpd.error_boxplot(sta_all,g = "time",s = {"member":["ob","ECMWF"]},
json_path=r"H:\test_data\output\mpd\program\json_error_boxplot.txt",
showfliers = False,vmax = 10) </code></pre>
<p><img src="https://www.showdoc.com.cn/server/api/attachment/visitfile/sign/bbf2709ad131aa470d363ca2407a4330" alt="" /></p>
<pre><code>have printed pictrue elements to H:\test_data\output\mpd\program\json_error_boxplot.txt</code></pre>
<pre><code class="language-python"></code></pre>