From 24481ade8b47bf497b8a2d36406b4c6f2646312c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kevin=20H=C3=B6llring?= <kevin.hoellring@fau.de>
Date: Mon, 25 Nov 2019 02:44:38 +0100
Subject: [PATCH] Finish CG method problem description
---
instructions/exercise_4.tex | 34 +++++++++++++++++++++++++---------
1 file changed, 25 insertions(+), 9 deletions(-)
diff --git a/instructions/exercise_4.tex b/instructions/exercise_4.tex
index 0a6b006..543fdf4 100644
--- a/instructions/exercise_4.tex
+++ b/instructions/exercise_4.tex
@@ -200,7 +200,8 @@ in your command line/shell before executing your program.
You usually additionally choose a limit to the number of iterations to avoid being trapped in an infinite loop due to the iteration diverging.\\
The class provides you with a member \code{stepsize} to be used as $a$ and a Differentiator (\code{diff}) which you can use to calculate the differential values of functions where required together with the differentiation step $h$ given by the member \code{diff\_precision}. These three things are not passed as arguments to the function \code{GradientDescent::optimize}.\\
The members \code{diff} and \code{diff\_precision} will be present in all three classes for optimization.\par
- Implement the iterated gradient descent as described in the method \code{GradientDescent::optimize}.
+ Implement the iterated gradient descent as described in the method \code{GradientDescent::optimize}. \par
+ (\emph{Note:} you can test your implementation using \code{make test\_gradient\_descent})
\end{homeworkProblem}
\begin{homeworkProblem}
An alternative to following the path of steepest descent towards a local minimum would be to use the fact that in a local optimum the gradient of the function one wants to optimize will vanish/become zero.
@@ -208,7 +209,8 @@ in your command line/shell before executing your program.
Implement the method \code{GradientRootfinder::optimize} where you determine a numerical approximation for a local extremum using the Newton method on the gradient of the function \code{func} starting at the intial position \code{start} until the iteration yields a precision of \code{precision} according to the previous definition of precision.\par
Note: You will need to provide the \code{findRoot} method in \code{NewtonRootFinder} with a function of appropriate format in order for the algorithm to work.
If you have not already done so for the previous exercise, take a look at \href{https://de.cppreference.com/w/cpp/language/lambda}{\code{C++}-lambda functions.} and the \class{LambdaWrapper} as provided in \path{include/lambda_wrapper.h}.
- If you want to skip this part of the exercise or in order to check that your implementation works, you can use the function \code{nabla} defined in \path{include/differential.h} to generate a function calculating the gradient of a function by providing both a \class{Function} object and a \class{Differentiator} object.
+ If you want to skip this part of the exercise or in order to check that your implementation works, you can use the function \code{nabla} defined in \path{include/differential.h} to generate a function calculating the gradient of a function by providing both a \class{Function} object and a \class{Differentiator} object.\par
+ (\emph{Note:} you can test your implementation using \code{make test\_gradient\_root})
\end{homeworkProblem}
\begin{homeworkProblem}
Last but not least we will have a look at the \emph{CG}-method of optimization which is related to the general idea of gradient descent.
@@ -219,9 +221,9 @@ in your command line/shell before executing your program.
As we want to approximate local minima of functions, a good choice for a matrix $A$ is the Hesse-Matrix of the funtion $f$ that we want to minimize at the position of the local minimum $x^\ast$.
Due to us not knowing the precise position of the minimum (why would we be looking to approximate it, if we were already able to locate it?) we will instead consider $A=\nabla^2f(x_k)$ the Hesse matrix at the current position to be an approximation.\\
Let us now describe the substep of the CG-method in more detail:
- \begin{enumerate}
+ \begin{enumerate}[label=\textbf{\arabic*}]
\item Let $x_0$ be the starting position. Calculate $g_0 = \nabla f(x_0)$ and choose the first direction $d_0 = -g_0$ as in the gradient descent.
- \item
+ \item
For $k=0,1,2,\ldots n-1$ with $n$ being the dimension of the parameter space of $f$, repeat the following:
\begin{enumerate}[label=\alph*)]
\item Calculate the Hesse matrix once and store its value:
@@ -238,11 +240,25 @@ in your command line/shell before executing your program.
$$ d_{k+1}= -g_{k+1}+\beta_k d_k$$
\end{enumerate}
\end{enumerate}
- The resulting $x_n$ is the end approximation of the substep and can afterwards be used as the input for subsequent executions of this subroutine.\par
- You will now implement the CG optimization algorithm in the method \code{optimize} of the class \class{ConjugateGradient}.
- To structure your code you will implement the method \code{CGstep} in that class, which executes one full iteration of the subroutine as detailed above.
+ The resulting $x_n$ is the end approximation of the substep and can afterwards be used as the input for subsequent executions of this subroutine.
\begin{enumerate}
- \item
- \end{enumerate}
+ \item In order to structure your code, implement the method \class{ConjugateGradient::CGstep}, which executes one full iteration of the subroutine as detailed above.\\
+ To do that you will need a few extra steps
+ \begin{enumerate}[label=\alph*)]
+ \item You need to be able to perform scalar products on vectors. Add and implement the overloaded multiplication operator
+ \begin{center}
+ \code{double operator*(const Vector<double> \&other) const;}
+ \end{center}
+ in the \class{Vector<T>}-class which returns the scalar value of the scalar product of the current \class{Vector<T>} and the other \class{Vector<T>}.
+ Do not change the signature from what is given above. Implement the method directly in the header file. (In case you are wondering why, it has something to do with the fact that \class{Vector<T>} uses a dynamic type parameter.)
+ \item You need to obtain the Hesse-matrix of a function.
+ You can obtain it by calculating the Jacobi-matrix of the gradient function of \code{func}.
+ You may be able to re-use your solution to the issue in problem 2 that the Newton-Raphson algorithm requires a \class{Function}-instance calculating the gradient of \code{func}.\\
+ Think of a way to obtain the Hesse-matrix in your program.
+ \end{enumerate}
+ Use these functionalities or ideas to implement the method \class{ConjugateGradient::CGstep}.
+ \item Proceed to implement the CG optimization algorithm in the method \class{optimize} of the class \class{ConjugateGradient}. Use your implementation of \class{ConjugateGradient::CGstep}.
+ \end{enumerate}\par
+ (\emph{Note:} you can test your implementation using \code{make test\_conjugate\_gradient})
\end{homeworkProblem}
\end{document}
--
GitLab