Unit information: Nonparametric Regression in 2012/13

Unit name	Nonparametric Regression
Unit code	MATHM6004
Credit points	10
Level of study	M/7
Teaching block(s)	Teaching Block 1A (weeks 1 - 6)
Unit director	Dr. Kovac
Open unit status	Not open
Pre-requisites	None
Co-requisites	None
School/department	School of Mathematics
Faculty	Faculty of Science

Description including Unit Aims

A regression function is an important tool for describing the relation between two or more random variables. In real life problems, this function is usually unknown but can be estimated from a sample of observations. In the most simple cases, we have enough information on the problem at hand to assume that the regression curve is known up to the value of some coefficients (for example, it is a straight line, but we need to estimate the coefficients of the line). Nonparametric methods are flexible techniques dedicated to treat more general cases: here, we construct a good estimator of the regression function without assuming that it has a specified shape. In this module, we will introduce popular nonparametric methods of regression estimation: local polynomial regression, regularisation techniques and wavelet thresholding. We will see how these methods can be applied in practice.

Aims

• Teach the problem of nonparametric function estimation
• Introduce several methods like kernel estimators and spline techniques
• Construct estimators of a signal from a sample of observations
• Learn flexible and popular techniques of distribution-free methods

Syllabus

• Introduction to nonparametric methods (explanation of differences with parametric methods)

• Kernel method: local polynomial regression as a generalization of the usual linear regression. Necessity of a smoothing parameter.

• Selection of the smoothing parameter in practice + some properties of local polynomial estimators.

• Spline regression as a generalization of the usual linear regression. Smoothing spline, penalty, smoothing parameter, properties of the spline estimators

• Brief introduction to wavelet regression

• Smoothing under Shape Constraints

Intended Learning Outcomes

The students will be able to:

• calculate nonparametric regression estimates in R

• decide on suitable techniques for choosing smoothing parameters involved in techniques for nonparametric regression

• compute the various nonparametric regression estimators for given data

• know the strengths and weaknesses of the various methods introduced

• be able to decide the most appropriate method to use given the data at hand.

Transferable Skills:

In addition to the general skills associated with other mathematical units, students will also have the opportunity to gain practice in the following: report writing, use of information resources, use of initiative in learning material in other than that provided by the lectures themselves, time management, general IT skills and word-processing.

Teaching Information

Lectures supported by exercise sheets, many of which involve computer practical work.

Assessment Information

Assessment will be by means of a project involving applications on the computer and some theoretical properties of the estimators.

Reading and References

The main references are:

• D. Ruppert, M. P. Wand, and R. J. Carroll (2003). Semiparametric Regression.
• Wand, M.P and Jones, M.C. (1995). Kernel smoothing.
• J. Fan and I. Gijbels. (1996) Local Polynomial Modeling and Its Applications. Chapman & Hall,London

but the following books can also be heplful:

• Bowman, A.W. and Azzalini, A. (1997), Applied smoothing techniques for data analysis
• Haerdle, W. (1991) Smoothing techniques : with implementation in S
• Hastie, T. Tibshirani, R. and Friedman, J. H. (2003). The Elements of Statistical Learning.