AI News, 13th in Mars Express Orbiter - European Space Agency - Machine Learning competition
13th in Mars Express Orbiter - European Space Agency - Machine Learning competition
The challenge was to predict the thermal power(supplies power to the platform units and the thermal subsystem) required for a martian year(which is approx 2 earth years) for the Mars Express Orbiter satellite.
Training data was provided for 3 martian years(2008 to 2014), test data for the year 2014.
SAAF: Context data contained the solar aspect angles, the (x,y,z) angles of the satellite with respect to the Sun.
LTDATA: Contained various distance related variables like the distance from earth to mars, distance from sun to mars.
extracted multiple features like Start, value, command, dyns event, occurence number and trigger from the command text and label encoder them.
fell down from 7 to 14 in the public leaderboard in the last few days and ended up 13th on the private leaderboard in the end.
Multivariate adaptive regression splines
In statistics, multivariate adaptive regression splines (MARS) is a form of regression analysis introduced by Jerome H.
Friedman in 1991. It is a non-parametric regression technique and can be seen as an extension of linear models that automatically models nonlinearities and interactions between variables.
We start with a set of data: a matrix of input variables x, and a vector of the observed responses y, with a response for each row in x.
The data at the extremes of x indicates that the relationship between y and x may be non-linear (look at the red dots relative to the regression line at low and high values of x).
MARS software constructs a model from the given x and y as follows The figure on the right shows a plot of this function: the predicted
In this simple example, we can easily see from the plot that y has a non-linear relationship with x (and might perhaps guess that y varies with the square of x).
However, in general there will be multiple independent variables, and the relationship between y and these variables will be unclear and not easily visible by plotting.
An example MARS expression with multiple variables is This expression models air pollution (the ozone level) as a function of the temperature and a few other variables.
To obtain the above expression, the MARS model building procedure automatically selects which variables to use (some variables are important, others not), the positions of the kinks in the hinge functions, and how the hinge functions are combined.
hinge function is zero for part of its range, so can be used to partition the data into disjoint regions, each of which can be treated independently.
Thus for example a mirrored pair of hinge functions in the expression creates the piecewise linear graph shown for the simple MARS model in the previous section.
One might assume that only piecewise linear functions can be formed from hinge functions, but hinge functions can be multiplied together to form non-linear functions.
At each step it finds the pair of basis functions that gives the maximum reduction in sum-of-squares residual error (it is a greedy algorithm).
Each new basis function consists of a term already in the model (which could perhaps be the intercept term) multiplied by a new hinge function.
A hinge function is defined by a variable and a knot, so to add a new basis function, MARS must search over all combinations of the following: 1) existing terms (called parent terms in this context) 2) all variables (to select one for the new basis function) 3) all values of each variable (for the knot of the new hinge function).
This process of adding terms continues until the change in residual error is too small to continue or until the maximum number of terms is reached.
The search at each step is done in a brute force fashion, but a key aspect of MARS is that because of the nature of hinge functions the search can be done relatively quickly using a fast least-squares update technique.
(An overfit model has a good fit to the data used to build the model but will not generalize well to new data.) To build a model with better generalization ability, the backward pass prunes the model.
The backward pass has an advantage over the forward pass: at any step it can choose any term to delete, whereas the forward pass at each step can only see the next pair of terms.
The forward pass adds terms in pairs, but the backward pass typically discards one side of the pair and so terms are often not seen in pairs in the final model.
The backward pass uses generalized cross validation (GCV) to compare the performance of model subsets in order to choose the best subset: lower values of GCV are better.
Such new data is usually not available at the time of model building, so instead we use GCV to estimate what performance would be on new data.
The raw residual sum-of-squares (RSS) on the training data is inadequate for comparing models, because the RSS always increases as MARS terms are dropped.
In other words, if the RSS were used to compare models, the backward pass would always choose the largest model—but the largest model typically does not have the best generalization performance.) The formula for the GCV is where RSS is the residual sum-of-squares measured on the training data and N is the number of observations (the number of rows in the x matrix).
We penalize flexibility because models that are too flexible will model the specific realization of noise in the data instead of just the systematic structure of the data.
- On Sunday, July 21, 2019
SPM Parameter Extraction with Marsbar
Demonstration of how to extract parameter estimates from multiple ROIs using Marsbar. Related blog post: ...
Bring AI to the mainstream with updates to Azure Machine Learning | B112
We take a look at updates to Azure Machine Learning that help you create and deliver powerful AI solutions. We show you how Machine Learning will help you ...
Amir Husain: "The Sentient Machine: The Coming Age of Artificial Intelligence" | Talks at Google
The Sentient Machine addresses broad existential questions surrounding the coming of AI: Why are we valuable? What can we create in this world? How are we ...
AI Planning for Space by Brad Clement
AI Planning for Space by Bradley J. Clement, Jet Propulsion Laboratory, NASA. Video production by Brad Clement and used with permission. Images from ...
Manuel Ebert - Putting 1 million new words into the dictionary - PyCon 2016
Speaker: Manuel Ebert 2015 was the year of spocking, amabots, dadbuds, and smol. Like half of all english words used every day, these words are not in the ...
Inside Exchange Online - BRK4029
At Ignite 2016, we showed you how we operate Exchange Online. This year, we're going one step further, to show you how Exchange Online works.
Council Meeting of April 4, 2016