Lecture 10 - Model Identification - Stanford University
Lecture 10 - Model Identification What is system identification?Direct impulse response identificationLinear regressionRegularizationParametric model ID, nonlinear LSEE392m - Spring 2005GorinevskyControl Engineering10-1
What is System tionModel White-box identification– estimate parameters of a physical model from data– Example: aircraft flight model Gray-box identificationRarely used inreal-life control– given generic model structure estimate parameters from data– Example: neural network model of an engine Black-box identification– determine model structure and estimate parameters from data– Example: security pricing models for stock marketEE392m - Spring 2005GorinevskyControl Engineering10-2
Industrial Use of System ID Process control - most developed ID approaches– all plants and processes are different– need to do identification, cannot spend too much time on each– industrial identification tools Aerospace– white-box identification, specially designed programs of tests Automotive– white-box, significant effort on model development and calibration Disk drives– used to do thorough identification, shorter cycle time Embedded systems– simplified models, short cycle timeEE392m - Spring 2005GorinevskyControl Engineering10-3
Impulse response identification Simplest approach: apply control impulse and collect thedataIMPULSE RESPONSE10.80.60.40.2001234567 Difficult to apply a short impulse big enough such that theresponse is much larger than the noiseTIMENOISY IMPULSE RESPONSE10.50-0.50123456TIME FIR modeling can be used for building simplified controldesign models from complex simsEE392m - Spring 2005GorinevskyControl Engineering10-47
Step response identification Step (bump) control input and collect the data– used in process controlActuator bumpedSTEP RESPONSE OF PAPER WEIGHT1.510.500200400600TIME (SEC)8001000 Impulse estimate: impulse(t) step(t)-step(t-1)IMPULSE RESPONSE OF PAPER WEIGHT Still noisy0.30.20.100EE392m - Spring 2005Gorinevsky100Control Engineering200300400TIME (SEC)50060010-5
Noise reductionNoise can be reduced by statistical averaging: Collect data for multiple step inputs and perform moreaveraging to estimate the step/pulse response Use a parametric model of the system and estimate a fewmodel parameters describing the response: dead time, risetime, gain Do both in a sequence– done in real process control ID packages Pre-filter dataEE392m - Spring 2005GorinevskyControl Engineering10-6
Linear Regression - univariateSTEP RESPONSE OF PAPER WEIGHT Simple fitting problem:1.5– Given model stepresponse y (t)– And process stepresponse ϕ (t)– Find the gain factor θy(t)1ϕ (t )0.500200y (t ) θϕ (t ) e(t )600TIME (SEC)8001000y Φθ e ϕ (1) e(1) y (1) y M , Φ M , e M ϕ ( N ) e( N ) y ( N ) EE392m - Spring 2005Gorinevsky400Control EngineeringSolution assuminguncorrelated noise:ΦT yθ TΦ Φ10-7
Linear Regression Linear regression is one of the main System ID toolsRegression weightsRegressorError of the fitDataKy (t ) θ jϕ j (t ) e(t )j 1y Φθ e y (1) ϕ1 (1) K ϕ K (1) θ1 e(1) OM , θ M , e M y M , Φ M y ( N ) ϕ1 ( N ) K ϕ K ( N ) θ K e( N ) EE392m - Spring 2005GorinevskyControl Engineering10-8
Linear regression - least squares Makes sense only when matrix Φ is tall,N K, more data available than thenumber of unknown parameters.– Statistical averaging Least square solution: e 2 minL ( y Φ θ ) ( y Φ θ ) min L 2Φ T ( y Φ θ ) 0 θT 1 TTˆθ (Φ Φ ) Φ y Can be computed using Matlab pinv or left matrix division \EE392m - Spring 2005GorinevskyControl Engineering10-9
Linear regression - least squares Correlation interpretation of the least squares solutionθˆ (Φ T Φ ) Φ T yθˆ R 1c 11R ΦT ΦN N 2K ϕ 1 (t )1 t 1MOR N N ϕ1 (t )ϕ K (t ) K t 1 ϕ K (t )ϕ1 (t ) t 1 M, N2 ϕ K (t ) t 1 NInformation matrixEE392m - Spring 2005Gorinevsky1 Tc Φ yN N ϕ1 ( t ) y ( t ) 1 t 1 Mc N N ϕ K (t ) y (t ) t 1 Correlation vectorControl Engineering10-10
Example: First-order ARMA modely (t ) ay (t 1) gu (t 1) e(t ) Linear regression representationϕ1 (t ) y (t 1) a y (t ) θ1ϕ1 (t ) θ 2ϕ 2 (t ) e(t )θ ϕ 2 (t ) u(t 1) g 1 TTˆθ (Φ Φ ) Φ y This (type of) approach is considered in most of the technicalliterature on identificationLennart Ljung, Matlab Identification ToolboxSystem Identification: Theory– Limited industrial usefor the User, 2nd Ed, 1999 Fundamental issue:– Small error in a might mean large change in the system responseEE392m - Spring 2005GorinevskyControl Engineering10-11
Regularization Linear regression, where Φ T Φ is ill-conditioned Instead of e 2 min solve a regularized problem2e rθ2 miny Φθ ewhere r is a small regularization parameter A.N.Tikhonov (1963)– see http://solon.cma.univie.ac.at/ neum/ms/regtutorial.pdf Regularized solution 1 TTˆθ (Φ Φ rI ) Φ y Cut off the singular values of Φ that are smaller than rEE392m - Spring 2005GorinevskyControl Engineering10-12
Regularization Analysis through SVD (singular value decomposition)Φ USVT Regularized solutionV R K ,K ; U R N ,K ; S diag{s j }Kj 1U T U VV T IK s 1 θˆ (Φ T Φ rI ) Φ T y V diag 2 j U T y s j r j 1 Cut off the singular values of Φ that are smaller than r210Inverse singular values 1/s1Regularized inversesvalues 2s 0 .1EE392m - Spring 2005GorinevskyINVERSE10010-110-210-210-110Control Engineering010SINGULAR VALUE1s1021010-13
Linear regression for FIR modelPRBS EXCITATION SIGNAL Identifying impulse response by 10.5applying multiple steps0-0.5 PRBS excitation signal-1 FIR (impulse response) model 0Ky ( t ) h ( k )u ( t k ) e( t )k 110203040PRBS Pseudo-Random Binary SequenceSee IDINPUT in Matlab Linear regression representationϕ1 (t ) u(t 1)u ( t 2)u(t K )K u(t 1) h(1) , θ M MMMOM,Φ ϕ K (t ) u (t K ) u(t N ) u(t N 1) K u(t N K 1) h( K ) 1 TTˆRegularized LS solution: θ (Φ Φ rI ) Φ yEE392m - Spring 2005GorinevskyControl Engineering10-1450
Example: FIR model IDPRBS excitation PRBS TEM RESPONSE Simulated system 1output: 40000.5samples, random 0-0.5noise of the-1amplitude 0.50400200400600TIMEEE392m - Spring 2005GorinevskyControl Engineering10-15
Example: FIR model IDFIR estimate Linear regressionestimate of the FIRmodel0.20.150.10.050-0.050 Impulse response forthe simulated system: 0.21234567567Impulse Response0.150.10.050-0.0501234Time (sec)H tf([1 .5],[1 1.1 1]);P c2d(H,0.25);EE392m - Spring 2005GorinevskyControl Engineering10-16
Nonlinear parametric model ID Prediction model depending onthe unknown parameter vector θu (t ) MODEL(θ ) yˆ (t θ ) Nonlinear regression: loss indexOptimizerθL y (t ) yˆ (t θ )L y (t ) yˆ (t θ ) miny (t ) Iterative numerical optimization.Computation of L as a subroutineu (t )2Loss Index LModel including theparameters θsimLennart Ljung, “Identification for Control: Simple Process Models,”IEEE Conf. on Decision and Control, Las Vegas, NV, 2002EE392m - Spring 2005GorinevskyControl Engineering210-17
Parametric SysID of step responseτ First order process with deadtime Most common industrial process model Response to a control step applied at tBgTD g (1 e( t tB TD ) /τ ), for t tB TDy (t θ ) γ 0,for t tB TD γ g θ τ TD Example:PapermachineprocessEE392m - Spring 2005GorinevskyControl Engineering10-18
Step1: Gain and Offset EstimationTwo-step approach: linear regression nonlinear regression For given τ , TD , the modeled step response can be presentedin the formy (t θ ;τ , TD ) γ g y1 (t τ , TD ) This is a linear regression2y (t θ ;τ , TD ) θ k ϕ k (t )k 1θ1 gθ2 γϕ1 (t ) y1 (t τ , TD )ϕ 2 (t ) 1 Parameter estimate and prediction for given τ , TD 1Tˆˆθ θ (τ , TD ) (Φ Φ ) Φ T yEE392m - Spring 2005Gorinevskyyˆ (t τ , TD ) γˆ gˆ y1 (t τ , TD )Control Engineering10-19
Step 2: Rise Time & Dead Time Estimation For any given τ , TD, the loss index isNL y (t ) yˆ (t τ , TD )2t 1 Grid τ , TD and find the minimum of L L(τ , TD )EE392m - Spring 2005GorinevskyControl Engineering10-20
Examples: Step Response ID Identification results for real industrial process data This algorithm works in an industrial tool used in 500 industrial plants, many processes each11.4NonlinearRegression ID0.80.61.210.40.8LinearRegression IDof the first-ordermodel0.20-0.2-0.4Process parameters: Gain 0.134; Tdel 0.00; Trise 119.89691.6010203040EE392m - Spring 2005Gorinevsky506070NonlinearRegression ID0.60.40.2080-0.20100200300400500600700time in sec.; MD response - solid; estimated response - dashedControl Engineering10-21800
Linear Filtering in SysIDu A trick that helps: pre-filter data Consider data modelPlantySysIDy h *u eĥ F is a linear filtering operator, usually LPFFy F ( h * u ) Fe{{eyfPlantuyFfF ( h * u ) ( Fh ) * u h * ( Fu )FSysIDĥ Can estimate h from filtered y and filtered u Or can estimate filtered h from filtered y and ‘raw’ u Pre-filter bandwidth limits the estimation bandwidthEE392m - Spring 2005GorinevskyControl Engineering10-22
Multivariable Identification Step/impulse response identification is a key part of theindustrial multivariable Model Predictive Control packages Apply SISO ID tovarious input/outputpairs Need n tests: exciteeach input in turnand collect alloutputs at thatEE392m - Spring 2005GorinevskyControl Engineering10-23
Industrial Use of System ID Process control - most developed ID approaches – all plants and processes are different – need to do identification, cannot spend too much time on each – industrial identification tools Aerospace – white-box identification, specially designed programs of tests Automotive
Introduction of Chemical Reaction Engineering Introduction about Chemical Engineering 0:31:15 0:31:09. Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Lecture 27 Lecture 28 Lecture
SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularity Qingyuan Zhao Stanford University qyzhao@stanford.edu Murat A. Erdogdu Stanford University erdogdu@stanford.edu Hera Y. He Stanford University yhe1@stanford.edu Anand Rajaraman Stanford University anand@cs.stanford.edu Jure Leskovec Stanford University jure@cs.stanford .
Lecture 1: A Beginner's Guide Lecture 2: Introduction to Programming Lecture 3: Introduction to C, structure of C programming Lecture 4: Elements of C Lecture 5: Variables, Statements, Expressions Lecture 6: Input-Output in C Lecture 7: Formatted Input-Output Lecture 8: Operators Lecture 9: Operators continued
Domain Adversarial Training for QA Systems Stanford CS224N Default Project Mentor: Gita Krishna Danny Schwartz Brynne Hurst Grace Wang Stanford University Stanford University Stanford University deschwa2@stanford.edu brynnemh@stanford.edu gracenol@stanford.edu Abstract In this project, we exa
Computer Science Stanford University ymaniyar@stanford.edu Madhu Karra Computer Science Stanford University mkarra@stanford.edu Arvind Subramanian Computer Science Stanford University arvindvs@stanford.edu 1 Problem Description Most existing COVID-19 tests use nasal swabs and a polymerase chain reaction to detect the virus in a sample. We aim to
Stanford University Stanford, CA 94305 bowang@stanford.edu Min Liu Department of Statistics Stanford University Stanford, CA 94305 liumin@stanford.edu Abstract Sentiment analysis is an important task in natural language understanding and has a wide range of real-world applications. The typical sentiment analysis focus on
Lecture 1: Introduction and Orientation. Lecture 2: Overview of Electronic Materials . Lecture 3: Free electron Fermi gas . Lecture 4: Energy bands . Lecture 5: Carrier Concentration in Semiconductors . Lecture 6: Shallow dopants and Deep -level traps . Lecture 7: Silicon Materials . Lecture 8: Oxidation. Lecture
TOEFL Listening Lecture 35 184 TOEFL Listening Lecture 36 189 TOEFL Listening Lecture 37 194 TOEFL Listening Lecture 38 199 TOEFL Listening Lecture 39 204 TOEFL Listening Lecture 40 209 TOEFL Listening Lecture 41 214 TOEFL Listening Lecture 42 219 TOEFL Listening Lecture 43 225 COPYRIGHT 2016
