{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import x3cflux"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "# Understanding the Simulator Object\n",
    "\n",
    "The simulator object is the core of 13CFLUX3. It redirects atomic performance critical computations, such as forward simulations and its derivative, to highly performant C++ code. Building on these atomic computations, higher level routines, such as optimization or statistics, are available via Python. This guide introduces how to best create simulator objects and gives a quick overview of its features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "## Creating simulator objects\n",
    "\n",
    "Due to technical reasons, creating a simulator does not directly translate to instantiation of a class and requires calling special functions which are handed the necessary data and compile the simulator. This may sound complicated, but is very straightforward in practice and combines incredible performance with both flexibility and convenience. This tutorial explains the different ways of creating a simulator object and when to use which."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### The standard way: FluxML-defined model\n",
    "In the most simple case, network and labeling data are already encoded in the XML-based modeling language *FluxML*. We encourage using *FluxML* to obtain reproducible models. A model can be created by a single function call supplying the location of the *FluxML* file and the name of the *FluxML configuration*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "simulator = x3cflux.create_simulator_from_fml(\"spiralus.fml\", \"ms+uptake\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "FluxML files can store labeling data from multiple experiment. To create a simulator from multiple experiments with 13CFLUX3, a list of experiment names instead of single one is supplied."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "simulator = x3cflux.create_simulator_from_fml(\"spiralus.fml\", [\"ms+uptake\", \"ms\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### The flexible way: Programming-defined model\n",
    "Using the Python interface of 13CFLUX3, metabolic network including atom transitions and isotope labeling experiments can be created programmatically. This can be used for contextual modification of the metabolic network and or generation of in-silico labeling experiments without needing to edit XML code. There are two ingredients necessary to define labeling experiments from code: an `x3cflux.NetworkData` object for metabolites and reactions (including atom transitions) and one or multiple `x3cflux.MeasurementConfiguration` objects for labeling experiment specification and data.\n",
    "\n",
    "*FluxML* files can be read in as data objects, too. By"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "parser = x3cflux.FluxMLParser()\n",
    "data = parser.parse(\"spiralus.fml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "a *FluxML* file is parsed into a data object containing metabolic network and all labeling experiments specified in the file. Now, a simulator can be created by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "simulator = x3cflux.create_simulator_from_data(data.network_data, data.configurations[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "Similarly, both objects supplied to the `create_simulator_from_data` function can be replaced by self-created or modified ones."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### The special ways: Application-specific convenience functions\n",
    "13CFLUX3 provides two special `create_simulator_*` functions. They are supposed to streamline certain standard applications which require juggling with complicated data structures to reduce repetitive code and errors.\n",
    "\n",
    "The first application is a very common theoretical task, in which perturbed data is generated to estimate parameters from it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "perturbed_data = x3cflux.compute_perturbed_measurements(simulator, [0.6, 1.0], num_samples=1)\n",
    "simulator = x3cflux.create_simulator_from_measurements(perturbed_data, simulator)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "The second application is based on experimental design. Given a set of mixtures for some of the input tracers, a simulator modeling experiments for arbitrarily many different tracer mixtures is created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "mixtures = x3cflux.parse_substrate_mixture_collections(\"mixture.xml\")\n",
    "substrate_mixtures, _ = x3cflux.compute_mixture_samples(mixtures, 10)\n",
    "simulator = x3cflux.create_simulator_from_inputs(simulator.network_data,\n",
    "                                                 simulator.configurations[0],\n",
    "                                                 substrate_mixtures)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parametrization\n",
    "When speaking about parameters, the simulator's main functionalities all take **free net flux/exchange flux/pool size parameters**, i.e. vectors from the solution spaces of the metabolic steady state equation $\\mathbf{S} \\cdot \\nu = \\mathbf{0}$ and other user-supplied equality constraints. The choice of which parameters are free can be controlled by adding variables to the `<simulation>` section of the FluxML document or by supplying a list of `x3cflux.ParameterEntries` to `x3cflux.MeasurementConfiguration`. When supplying none or less than the systems DOF, free parameters will be selected automatically."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With a simulator at hand, parameter names can be easily accessed via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['q.n', 'u.n']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simulator.parameter_space.free_parameter_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "If supplied in the FluxML file or the `x3cflux.MeasurementConfiguration`, pre-computed configurations of free parameters can be obtained by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = x3cflux.get_parameters(simulator.parameter_space, simulator.configurations[0].parameter_entries)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the values of all parameters are desired, they can be computed by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('u.n', np.float64(1.0)),\n",
       " ('p.n', np.float64(0.4)),\n",
       " ('q.n', np.float64(0.6)),\n",
       " ('v.n', np.float64(0.4)),\n",
       " ('w.n', np.float64(0.4)),\n",
       " ('r.n', np.float64(0.6)),\n",
       " ('h_ouz.n', np.float64(0.6)),\n",
       " ('g_ouz.n', np.float64(0.4)),\n",
       " ('f_out.n', np.float64(0.4)),\n",
       " ('u.x', np.float64(0.0)),\n",
       " ('p.x', np.float64(0.0)),\n",
       " ('q.x', np.float64(0.0)),\n",
       " ('v.x', np.float64(0.0)),\n",
       " ('w.x', np.float64(0.0)),\n",
       " ('r.x', np.float64(0.0)),\n",
       " ('h_ouz.x', np.float64(0.0)),\n",
       " ('g_ouz.x', np.float64(0.0)),\n",
       " ('f_out.x', np.float64(0.0))]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(simulator.parameter_space.parameter_names, simulator.parameter_space.compute_parameters(params)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "## Functionalities\n",
    "The simulator object provides functionality to\n",
    "- Compute labeling simulations and its derivative (Jacobian)\n",
    "- Compute residuals to measurement data with exact first order (gradient) and approximated second order derivative (linearized hessian)\n",
    "\n",
    "which is the basis of all computational procedures from 13C-MFA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 37263.10013717, -22357.8600823 ],\n",
       "       [-22357.8600823 ,  13414.71604938]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simulator.compute_measurements(params)\n",
    "simulator.compute_jacobian(params)\n",
    "simulator.compute_loss(params)\n",
    "simulator.compute_loss_gradient(params)\n",
    "simulator.compute_linearized_hessian(params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Forward simulation, residual computation and residual gradients are capable of parallel computation for multiple parameter vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([12022.71022298, 12022.71022298])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "many_params = np.column_stack((params, params))\n",
    "simulator.compute_loss(many_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, all functions are available for single and multiple experiments. For multiple experiments, the above-mentioned functions provide special algorithms to speed up computations and generates combined output (e.g. `compute_loss` will return the sum of losses over all experiments). Special `compute_multi_...` (`jacobians`, `losses`, `loss_gradients`, `linearized_hessians`)functions on the other hand will return lists with the individual results of each experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1.3096323621833204e-28, 1.3096323621833204e-28]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x3cflux.create_simulator_from_fml(\"spiralus.fml\", [\"ms+uptake\", \"ms\"]).compute_multi_losses(params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The solution space is typically restricted by linear inequality constraints (e.g. from bounds on all parameters), given by $\\mathbf{C} \\cdot \\theta \\leq \\mathbf{d}$. Matrix and vector from this equation are accessible via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[-1.,  0.],\n",
       "        [ 0., -1.],\n",
       "        [ 1.,  0.],\n",
       "        [ 0.,  1.],\n",
       "        [ 1., -1.],\n",
       "        [ 1., -1.],\n",
       "        [ 1., -1.],\n",
       "        [ 1., -1.],\n",
       "        [ 1., -1.],\n",
       "        [ 1., -1.]]),\n",
       " array([ 0.,  0.,  5.,  5.,  0.,  0.,  0., -0.,  0.,  0.]))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ineq_sys = simulator.parameter_space.inequality_system\n",
    "ineq_sys.matrix, ineq_sys.bound"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is worth mentioning, that by default all above-mentioned functionalities require `params` to perfectly fulfill these inequality constraints. As computation might become mathematically unstable, it is however possibly to set a threshold, by which constraints may be violated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "simulator.parameter_space.constraint_violation_tolerance = 1e-6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Access of internal state\n",
    "The simulator object allows for easy access of internal states, such as parameter settings (`simulator.parameter_space`), reduced network (`simulator.network`) and labeling system builder (`simulator.builder`). By accessing the internal states, additional information about the inner working can be retrieved. Read the API for more in-depth understanding of how these classes store information. For technical reasons, each of these states map to slightly different classes depending on the characteristics of the experiment. E.g. in our current example, parameter setting are stored in the type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<x3cflux.lib.core.StationaryParameterSpace at 0x701a29902030>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simulator.parameter_space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For INST-MFA, on the other hand, `simulator.parameter_space` is an instance of `x3cflux.NonStationaryParameterSpace`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}