1 Regularized Nonlinear Regression with Dependent Errors and its Application to a Biomechanical Model

2025-04-30 0 0 1.66MB 28 页 10玖币

侵权投诉

Regularized Nonlinear Regression with Dependent Errors

and its Application to a Biomechanical Model

Hojun You1, Kyubaek Yoon3, Wei-Ying Wu4, Jongeun Choi3and Chae Young Lim2

1University of Houston 2Seoul National University 3Yonsei University 4National Dong Hwa University

Abstract: A biomechanical model often requires parameter estimation and selection in a

known but complicated nonlinear function. Motivated by observing that the data from a

head-neck position tracking system, one of biomechanical models, show multiplicative time

dependent errors, we develop a modiﬁed penalized weighted least squares estimator. The

proposed method can be also applied to a model with possible non-zero mean time dependent

additive errors. Asymptotic properties of the proposed estimator are investigated under

mild conditions on a weight matrix and the error process. A simulation study demonstrates

that the proposed estimation works well in both parameter estimation and selection with

time dependent error. The analysis and comparison with an existing method for head-neck

position tracking data show better performance of the proposed method in terms of the

variance accounted for (VAF).

Key words and phrases: nonlinear regression; temporal dependence; multiplicative error; local

consistency and oracle property

1. Introduction

A nonlinear regression model has been widely used to describe complicated relation-

ships between variables (Wood, 2010; Baker and Foley, 2011; Paula et al., 2015; Lim

et al., 2014; Salamh and Wang, 2021). In particular, various nonlinear problems

are considered in the ﬁeld of machinery and biomechanical engineering (Moon et al.,

2012; Santos and Barreto, 2017). Among such nonlinear problems, a head-neck po-

sition tracking model with neurophysiological parameters in biomechanics motivated

us to develop an estimation and selection method for a nonlinear regression model in

this work.

The head-neck position tracking application aims to ﬁgure out how characteristics

of the vestibulocollic and cervicocollic reﬂexes (VCR and CCR) contribute to the

head-neck system. The VCR activates neck muscles to stabilize the head-in-space

and the CCR acts to hold the head on the trunk. A subject of the experiment follows

a reference signal on a computer screen with his or her head and a head rotation angle

is measured during the experiment. A reference signal is the input of the system and

the measured head rotation angle is the output. The parameters related to VCR and

CCR in this nonlinear system are of interest to understand the head-neck position

tracking system.

The head-neck position problem has been widely studied in the literature of

biomechanics (Peng et al., 1996; Chen et al., 2002; Forbes et al., 2013; Ramadan et al.,

arXiv:2210.13550v2 [stat.ME] 11 Oct 2023

2018; Yoon et al., 2022). One of the prevalent issues in biomechanics is that a model

suﬀers from a relatively large number of parameters and limited availability of data

because the subjects in the experiment cannot tolerate suﬃcient time without being

fatigued. This leads to overﬁtting as well as non-identiﬁability of the parameters. To

resolve this issue, selection approaches via a penalized regression method have been

implemented to ﬁx a subset of the parameters to the pre-speciﬁed values while the

remaining parameters are estimated (Ramadan et al., 2018; Yoon et al., 2022).

Figure 1: The black curve represents the measured responses (the observations) from the subject

No. 8 in the head-neck position tracking experiment. The red dashed curve represents the estimated

responses (the ﬁtted values) from the nonlinear regression model with additive errors introduced in

Yoon et al. (2022).

Figure 2: Sample autocorrelation (left) and sample partial autocorrelation (right) of the residuals

(measured response−estimated response) for the subject No. 8 from the head-neck position tracking

experiment. The estimated response is obtained from the method in Yoon et al. (2022).

The existing approaches, however, have some limitations. The ﬁtted values from

the penalized nonlinear regression with additive errors in Yoon et al. (2022) show

larger discrepancy from the observed values when the head is turning in its direction.

For example, Figure 1 shows the ﬁtted values (the estimated responses) from the

method in Yoon et al. (2022) and the observations (the measured responses) of the

subject No. 8 in a head-neck position tracking experiment. The detailed description

of the data from the experiment is given in Section 4. We can observe that a larger

measured response leads to a bigger gap between the measured response and the

estimated response. Note that the largest measured responses occur when the head

is turning in its direction to follow the reference signal’s direction changes. It is more

diﬃcult to track the end positions correctly for some subjects, which can create more

errors at each end. Hence, the additive error structure considered in Yoon et al.

(2022) can not properly explain the data. Indeed, Figure 1 shows the model with

additive errors did not successfully accommodate this characteristic.

Another point we pose in this study is temporal dependence of the data which

previous studies did not also take into account while the experimental data exhibit

temporal correlation. For example, Figure 2 shows sample autocorrelation function

and sample partial autocorrelation function of the residuals from the ﬁtted model

for the subject No. 8 by the method in Yoon et al. (2022). The residuals clearly

show temporal dependence while Yoon et al. (2022) worked under independent er-

ror assumption. Lastly, Ramadan et al. (2018) and Yoon et al. (2022) restrict the

number of sensitive parameters to ﬁve, where the sensitive parameters refer to the

parameters whose estimates are not shrunk to the pre-speciﬁed values. Not only may

this restriction increase computational instability but also reduce estimation and pre-

diction performances. Provided the head-neck position tracking task already suﬀers

from computational challenges, additional computational issues should be avoided.

To resolve the above-mentioned issues, we consider a nonlinear regression model

with multiplicative errors for the head-neck position tracking system, which can be

written as

zt=g(xt;θ)×ςt,(1.1)

where ztis a measured response, g(x;θ) is a known nonlinear function with an

input x.θis a set of parameters in gand ςtis a multiplicative error. The details

for gand θfor the head-neck position tracking system are described in Ramadan

et al. (2018) and Yoon et al. (2022). The multiplicative error structure can better

explain the data than the previous studies in that the error may increase as the

signal increases. It is of importance that we choose suitable regression model for

data because Bhattacharyya et al. (1992) showed that an ordinary least squares

estimator for a nonlinear regression model with additive errors may not possess strong

consistency when the true underlying model has multiplicative errors.

A typical approach for the multiplicative error model is to take the logarithm

in both sides of (1.1) so that the resulting model becomes a nonlinear model with

additive errors:

yt=f(xt;θ) + ϵt,(1.2)

with yt= log(zt), f(xt;θ) = log(g(xt;θ)) and ϵt= log(ςt) by assuming all com-

ponents are positive. Note that the diﬀerence from the classical additive regression

model in Yoon et al. (2022) is that the additive error ϵtin (1.2) may have non-zero

mean due to the log transformation. By the relationship f= log(g), the conditions

imposed on gare inherited from the conditions on f. Therefore, the degree of ﬂexi-

bility allowed for gis linked to the ﬂexibility allowed for fbased on the assumptions

outlined in Section 2.

Motivated by the head-neck position tracking application, we propose a param-

eter estimation method for a nonlinear model with multiplicative error in (1.1) by

applying a modiﬁed weighted least squares estimation method to (1.2) which ac-

commodate temporal dependence as well as non-zero mean errors. Given by the

structure, the proposed estimation can also handle the nonlinear regression model

with possible non-zero mean additive errors in (1.2). The asymptotic properties of

the estimator obtained from the proposed method are studied under the assumption

of temporally correlated errors as we observed temporal dependence in the head-

neck position tracking system data. Introducing an additional intercept for non-zero

mean error could be a simple solution for handling (1.2). However, this approach

introduces an extra parameter to the original model, which could lead to ineﬃciency

in the estimation process. Indeed, as detailed in Section 3, our proposed approach

shows better performance in the simulation study for most cases and the diﬀerence

is apparent, in particular, when the non-zero mean is large or temporal dependence

is stronger. Furthermore, if the function f(xt;θ) already includes an intercept term,

the two intercept parameters are not identiﬁable.

A penalized estimator and its asymptotic properties are also investigated. In the

application of the head-neck position tracking system, a set of parameters needs to

be shrunk to the pre-speciﬁed values instead of the zero-values. Thus, we use the

penalized estimation approach to shrink estimates to the pre-speciﬁed values. By

doing so, we not only resolve the non-identiﬁability issue but also keep all estimates

meaningful in the head-neck position tracking system. For a penalty function, we

allow a general penalty function with mild conditions for the asymptotic properties

of the penalized least squares estimators. In the numerical study, least absolute

shrinkage and selection operator (LASSO, Tibshirani, 1996) and smoothly clipped

absolute deviation (SCAD, Fan and Li, 2001) are considered for results.

Numerous studies have explored the properties of least squares estimators in

nonlinear regression models. Jennrich (1969) ﬁrst proved the strong consistency and

asymptotic normality of the nonlinear least squares estimator with independent er-

rors. Wu (1981) provided necessary conditions for the existence of any kind of weakly

consistent estimators and extended the results in Jennrich (1969) under weaker con-

ditions. Pollard and Radchenko (2006) established asymptotic properties for least

squares estimators under second-moment assumptions for errors. Later, several stud-

ies (Wang and Leblanc, 2008; Kim and Ma, 2012; Ivanov et al., 2015; Radchenko,

2015; Salamh and Wang, 2021; Wang, 2021) delved further into the properties of the

least squares estimator in nonlinear regression.

There are several studies on the nonlinear regression with multiplicative errors.

Xu and Shimada (2000) studied least squares estimation for nonlinear multiplicative

noise models with independent errors, but their proposed estimator induces a bias

and needs correction. Lim et al. (2014) also investigated the nonlinear multiplica-

tive noise models with independent errors by the log transformation and proposed the

modiﬁed least square estimation by including a sample mean component in the objec-

tive function. Chen et al. (2010, 2016) proposed least absolute relative error (LARE)

and least product relative error (LPRE), respectively, as alternatives to least squares

for multiplicative regression. Zhang et al. (2022) further devised maximum nonpara-

metric kernel likelihood estimation for multiplicative regression. However, proposed

methods in Chen et al. (2010, 2016) and Zhang et al. (2022) can only accommodate

exponential nonlinear function, which highly limit the applicability of the methods.

Our study follows a similar approach to Lim et al. (2014) but least squares estima-

tion is enhanced with a weight matrix and temporally dependent errors are taken

into account in addition to the penalization.

To address temporal dependence in the errors, we consider mixing conditions, in-

cluding strong mixing (α-mixing), ϕ-mixing, and ρ-mixing, which are well-established

techniques for handling temporal data dependencies. Prior research has explored

various regression models involving mixing errors. Zhang and Liang (2012) studied

a semi-parametric regression model with strong mixing errors and established the

asymptotic normality of a weighted least squares estimator. Subsequently, Guo and

Liu (2019) devised wavelet regression estimators under strong mixing data and Ul-

lah et al. (2022) investigated the asymptotic properties of nonlinear modal estimator

with strong mixing errors. For more articles that accommodate mixing conditions in

regression problems, we refer to El Machkouri et al. (2017); Almanjahie et al. (2022);

Kurisu (2022); Mokhtari et al. (2022). In our study, the asymptotic properties of the

proposed estimator are established under various mixing conditions.

In Section 2, we demonstrate the proposed estimation method for a nonlinear

regression model and establish the asymptotic properties of the proposed estimators.

In Section 3, several simulation studies are conducted with various settings. The

analysis on head-neck position tracking data with the proposed method is introduced

in Section 4. At last, we provide a discussion in Section 5. The technical proofs

for the theorems and additional results of the simulation studies are presented in a

supplementary material.

2. Methods

2.1 Modiﬁed Weighted Least Squares

We consider a following nonlinear regression model

yt=f(xt;θ) + ϵt,

for t= 1,· · · , n, where xt∈D⊂Rdis a ﬁxed covariate vector and f(x;θ) is a

known nonlinear function on x, which also depends on the parameter vector θ:=

(θ1, θ2, ..., θp)T∈Θ. ATis the transpose of a matrix A.ϵtis temporally correlated

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

1RegularizedNonlinearRegressionwithDependentErrorsanditsApplicationtoaBiomechanicalModelHojunYou1,KyubaekYoon3,Wei-YingWu4,JongeunChoi3andChaeYoungLim21UniversityofHouston2SeoulNationalUniversity3YonseiUniversity4NationalDongHwaUniversityAbstract:Abiomechanicalmodeloftenrequiresparameterestimationan...

展开>> 收起<<

1 Regularized Nonlinear Regression with Dependent Errors and its Application to a Biomechanical Model.pdf

共28页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

1 Regularized Nonlinear Regression with Dependent Errors and its Application to a Biomechanical Model

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: