.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/plot_linear_regression.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_plot_linear_regression.py: Simple linear regression ============================ A simple linear regression example with two model parameters and one noise parameter. The model equation is y = a * x + b with a, b being the model parameters and the noise model is a normal zero-mean distribution with the std. deviation to infer. The problem is solved via sampling using emcee and pyro. .. GENERATED FROM PYTHON SOURCE LINES 13-14 Import what we will need for this example. .. GENERATED FROM PYTHON SOURCE LINES 14-23 .. code-block:: default import matplotlib.pyplot as plt import numpy as np from probeye.definition.forward_model import ForwardModelBase from probeye.definition.inference_problem import InferenceProblem from probeye.definition.noise_model import NormalNoiseModel from probeye.definition.sensor import Sensor from probeye.inference.emcee_.solver import EmceeSolver .. GENERATED FROM PYTHON SOURCE LINES 24-27 We start by generating a synthetic data set from a known linear model. Later we will pretend to forgot about the parameters of this ground truth model and will try to recover them from the data. The slope and intercept of the ground truth model: .. GENERATED FROM PYTHON SOURCE LINES 27-31 .. code-block:: default a_true = 2.5 b_true = 1.7 .. GENERATED FROM PYTHON SOURCE LINES 32-33 Generate a few data points that we contaminate with a Gaussian error: .. GENERATED FROM PYTHON SOURCE LINES 33-42 .. code-block:: default n_tests = 50 seed = 1 np.random.seed(seed) x_test = np.linspace(0.0, 1.0, n_tests) y_true = a_true * x_test + b_true sigma_noise = 0.5 y_test = y_true + np.random.normal(loc=0, scale=sigma_noise, size=n_tests) .. GENERATED FROM PYTHON SOURCE LINES 43-44 Visualize our data points .. GENERATED FROM PYTHON SOURCE LINES 44-49 .. code-block:: default plt.plot(x_test, y_test, "o") plt.xlabel("x") plt.ylabel("y") plt.show() .. image-sg:: /auto_examples/images/sphx_glr_plot_linear_regression_001.png :alt: plot linear regression :srcset: /auto_examples/images/sphx_glr_plot_linear_regression_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 50-51 Define our parametrized linear model: .. GENERATED FROM PYTHON SOURCE LINES 51-76 .. code-block:: default class LinearModel(ForwardModelBase): def response(self, inp): # this method *must* be provided by the user x = inp["x"] m = inp["m"] b = inp["b"] response = {} for os in self.output_sensors: response[os.name] = m * x + b return response def jacobian(self, inp): # this method *can* be provided by the user; if not provided # the jacobian will be approximated by finite differences x = inp["x"] # vector one = np.ones((len(x), 1)) jacobian = {} for os in self.output_sensors: # partial derivatives must only be stated for the model # parameters; all other input must be flagged by None; # note: partial derivatives must be given as column vectors jacobian[os.name] = {"x": None, "m": x.reshape(-1, 1), "b": one} return jacobian .. GENERATED FROM PYTHON SOURCE LINES 77-82 Define the inference problem. Initialize the inference problem with a useful name; note that the name will only be stored as an attribute of the InferenceProblem and is not important for the problem itself; can be useful when dealing with multiple problems .. GENERATED FROM PYTHON SOURCE LINES 82-85 .. code-block:: default problem = InferenceProblem("Linear regression with normal noise") .. rst-class:: sphx-glr-script-out Out: .. code-block:: none 2021-11-15 13:52:02.403 | INFO | # ================================================================================================ # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.404 | INFO | # # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.404 | INFO | # dP # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.404 | INFO | # 88 # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.404 | INFO | # 88d888b. 88d888b..d8888b. 88d888b. .d8888b. dP dP .d8888b. # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.405 | INFO | # 88' `88 88' 88' `88 88' `88 88ooood8 88 88 88ooood8 # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.405 | INFO | # 88. .88 88 88. .88 88. .88 88. 88. .88 88. # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.405 | INFO | # 88Y888P' dP `88888P' 88Y8888' `88888P' `8888P88 `88888P' # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.405 | INFO | # 88 .88 # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.405 | INFO | # dP d8888P # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.406 | INFO | # # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.406 | INFO | # ================================================================================================ # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.406 | INFO | # # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.406 | INFO | # Version 1.0.11 - A general framework for setting up parameter estimation problems. # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.406 | INFO | # # | probeye.subroutines:print_probeye_header:616 2021-11-15 13:52:02.406 | INFO | # ================================================================================================ # | probeye.subroutines:print_probeye_header:616 .. GENERATED FROM PYTHON SOURCE LINES 86-96 Add all parameters to the problem; the first argument states the parameter's global name (here: 'a', 'b' and 'sigma'); the second argument defines the parameter type (three options: 'model' for parameter's of the forward model, 'prior' for prior parameters and 'noise' for parameters of the noise model); the 'info'-argument is a short description string used for logging, and the tex-argument gives a tex-string of the parameter used for plotting; finally, the prior- argument specifies the parameter's prior; note that this definition of a prior will result in the initialization of constant parameters of type 'prior' in the background .. GENERATED FROM PYTHON SOURCE LINES 96-118 .. code-block:: default problem.add_parameter( "a", "model", tex="$a$", info="Slope of the graph", prior=("normal", {"loc": 2.0, "scale": 1.0}), ) problem.add_parameter( "b", "model", info="Intersection of graph with y-axis", tex="$b$", prior=("normal", {"loc": 1.0, "scale": 1.0}), ) problem.add_parameter( "sigma", "noise", tex=r"$\sigma$", info="Std. dev, of 0-mean noise model", prior=("uniform", {"low": 0.1, "high": 0.8}), ) .. GENERATED FROM PYTHON SOURCE LINES 119-133 Add the forward model to the problem; note that the first positional argument [{'a': 'm'}, 'b'] passed to LinearModel defines the forward model's parameters by name via a list with elements structured like {: }; a global name is a name introduced by problem.add_parameter, while a local name is a name used in the response-method of the forward model class (see the class LinearModel above); note that the use of the local parameter name 'm' for the global parameter 'a' is added here only to highlight the possibility of this feature; it is not necessary at all here; whenever forward model's parameter has a similar local and global name (which should be the case most of the times), one doesn't have to use the verbose notation {: } but can instead just write the parameter's (global=local) name, like it is done with the forward model's parameter 'b' below .. GENERATED FROM PYTHON SOURCE LINES 133-141 .. code-block:: default isensor = Sensor("x") osensor = Sensor("y") linear_model = LinearModel([{"a": "m"}, "b"], [isensor], [osensor]) problem.add_forward_model("LinearModel", linear_model) # add the noise model to the problem problem.add_noise_model(NormalNoiseModel(prms_def={"sigma": "std"}, sensors=osensor)) .. GENERATED FROM PYTHON SOURCE LINES 142-143 Add test data to the Inference Problem .. GENERATED FROM PYTHON SOURCE LINES 143-153 .. code-block:: default problem.add_experiment( "TestSeries_1", fwd_model_name="LinearModel", sensor_values={isensor.name: x_test, osensor.name: y_test}, ) # give problem overview problem.info() .. rst-class:: sphx-glr-script-out Out: .. code-block:: none 2021-11-15 13:52:02.413 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.413 | INFO | Problem summary: Linear regression with normal noise | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.413 | INFO | ==================================================== | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.414 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.414 | INFO | Forward models | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.414 | INFO | --------------------------------------------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.414 | INFO | Model name | Global parameters | Local parameters | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.414 | INFO | --------------+---------------------+-------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.414 | INFO | LinearModel | a, b | m, b | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.415 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.415 | INFO | Priors | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.415 | INFO | ----------------------------------------------------------------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.415 | INFO | Prior name | Global parameters | Local parameters | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.415 | INFO | ---------------+------------------------------+------------------------------ | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.415 | INFO | a_normal | a, loc_a, scale_a | a, loc_a, scale_a | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.416 | INFO | b_normal | b, loc_b, scale_b | b, loc_b, scale_b | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.416 | INFO | sigma_uniform | sigma, low_sigma, high_sigma | sigma, low_sigma, high_sigma | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.416 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.416 | INFO | Parameter overview | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.416 | INFO | ----------------------------------------------------------------------------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.416 | INFO | Parameter type/role | Parameter names | Count | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.417 | INFO | -----------------------+-------------------------------------------------------+--------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.417 | INFO | Model parameters | a, b | 2 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.417 | INFO | Prior parameters | loc_a, scale_a, loc_b, scale_b, low_sigma, high_sigma | 6 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.417 | INFO | Noise parameters | sigma | 1 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.417 | INFO | Const parameters | loc_a, scale_a, loc_b, scale_b, low_sigma, high_sigma | 6 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.417 | INFO | Latent parameters | a, b, sigma | 3 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.418 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.418 | INFO | Parameter explanations | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.418 | INFO | --------------------------------------------------------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.418 | INFO | Name | Short explanation | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.418 | INFO | ------------+-------------------------------------------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.418 | INFO | loc_a | Normal prior's parameter for latent parameter 'a' | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.419 | INFO | scale_a | Normal prior's parameter for latent parameter 'a' | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.419 | INFO | a | Slope of the graph | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.419 | INFO | loc_b | Normal prior's parameter for latent parameter 'b' | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.419 | INFO | scale_b | Normal prior's parameter for latent parameter 'b' | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.419 | INFO | b | Intersection of graph with y-axis | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.419 | INFO | low_sigma | Uniform prior's parameter for latent parameter 'sigma' | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.420 | INFO | high_sigma | Uniform prior's parameter for latent parameter 'sigma' | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.420 | INFO | sigma | Std. dev, of 0-mean noise model | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.420 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.420 | INFO | Constant parameters | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.420 | INFO | ---------------------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.420 | INFO | Name | Value | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.421 | INFO | ------------+--------- | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.421 | INFO | loc_a | 2 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.421 | INFO | scale_a | 1 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.421 | INFO | loc_b | 1 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.421 | INFO | scale_b | 1 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.422 | INFO | low_sigma | 0.1 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.422 | INFO | high_sigma | 0.8 | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.422 | INFO | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.422 | INFO | Theta interpretation | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.422 | INFO | +---------------------------+ | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.422 | INFO | | Theta | Parameter | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.423 | INFO | | index | name | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.423 | INFO | |---------------------------| | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.423 | INFO | | 0 --> a | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.423 | INFO | | 1 --> b | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.423 | INFO | | 2 --> sigma | | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.423 | INFO | +---------------------------+ | probeye.definition.inference_problem:info:267 2021-11-15 13:52:02.424 | INFO | | probeye.definition.inference_problem:info:267 "\nProblem summary: Linear regression with normal noise\n====================================================\n\nForward models\n---------------------------------------------------------\n Model name | Global parameters | Local parameters\n--------------+---------------------+--------------------\n LinearModel | a, b | m, b\n\nPriors\n-----------------------------------------------------------------------------\n Prior name | Global parameters | Local parameters\n---------------+------------------------------+------------------------------\n a_normal | a, loc_a, scale_a | a, loc_a, scale_a\n b_normal | b, loc_b, scale_b | b, loc_b, scale_b\n sigma_uniform | sigma, low_sigma, high_sigma | sigma, low_sigma, high_sigma\n\nParameter overview\n-----------------------------------------------------------------------------------------\n Parameter type/role | Parameter names | Count\n-----------------------+-------------------------------------------------------+---------\n Model parameters | a, b | 2\n Prior parameters | loc_a, scale_a, loc_b, scale_b, low_sigma, high_sigma | 6\n Noise parameters | sigma | 1\n Const parameters | loc_a, scale_a, loc_b, scale_b, low_sigma, high_sigma | 6\n Latent parameters | a, b, sigma | 3\n\nParameter explanations\n---------------------------------------------------------------------\n Name | Short explanation\n------------+--------------------------------------------------------\n loc_a | Normal prior's parameter for latent parameter 'a'\n scale_a | Normal prior's parameter for latent parameter 'a'\n a | Slope of the graph\n loc_b | Normal prior's parameter for latent parameter 'b'\n scale_b | Normal prior's parameter for latent parameter 'b'\n b | Intersection of graph with y-axis\n low_sigma | Uniform prior's parameter for latent parameter 'sigma'\n high_sigma | Uniform prior's parameter for latent parameter 'sigma'\n sigma | Std. dev, of 0-mean noise model\n\nConstant parameters\n----------------------\n Name | Value\n------------+---------\n loc_a | 2\n scale_a | 1\n loc_b | 1\n scale_b | 1\n low_sigma | 0.1\n high_sigma | 0.8\n\nTheta interpretation\n+---------------------------+\n| Theta | Parameter |\n| index | name |\n|---------------------------|\n| 0 --> a |\n| 1 --> b |\n| 2 --> sigma |\n+---------------------------+\n" .. GENERATED FROM PYTHON SOURCE LINES 154-155 Estimate the parameters using `emcee` .. GENERATED FROM PYTHON SOURCE LINES 155-157 .. code-block:: default emcee_solver = EmceeSolver(problem, show_progress=True) inference_data = emcee_solver.run_mcmc(n_walkers=20, n_steps=2_000, n_initial_steps=200) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none 2021-11-15 13:52:02.426 | INFO | Solving problem using emcee sampler with 200 + 2000 samples and 20 walkers | probeye.inference.emcee_.solver:run_mcmc:113 2021-11-15 13:52:02.426 | INFO | No additional options specified | probeye.inference.emcee_.solver:run_mcmc:120 0%| | 0/200 [00:00` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_linear_regression.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_