$\newcommand{\vp}{\varphi}$$\newcommand{\dmux}{\,\text d\mu(x)}$$\newcommand{\dmu}{\,\text d\mu}$$\newcommand{\E}{\operatorname{E}}$Let $\mu$ be a probability measure on $(\mathbb R, \mathbb B)$ where $\mathbb B$ is the Borel $\sigma$-algebra over $\mathbb R$ and define a function $\vp : \mathbb R \to \mathbb C$ by
$$
\vp(t) = \int_{\mathbb R} e^{itx} \dmux.
$$
$\vp$ is the characteristic function of $\mu$. I’ll be doing all of this just using the measure $\mu$ rather than using random variables, which is why I’m not using the more common definition of $\vp(t) = \E(e^{itX})$ for some random variable $X$ with $\mu$ as its distribution. This also makes it obvious that characteristic functions in probability are just Fourier transforms of finite measures.
Result 1: for a Borel $f : \mathbb R \to \mathbb R$, if $f$ is almost surely bounded then $\int f\dmu < \infty$.
Pf: $f$ being almost surely bounded means there is some $M \geq 0$ such that $\mu\left(\{|f| > M\}\right) = 0$ (also $f$ being Borel means $\{|f| > M\}$ is measurable).
I can split $f$ into its positive and negative parts via
$$
f = f_+ – f_-
$$
for $f_+ = \max\{f, 0\}$ and $f_- = \max\{-f, 0\}$. Because $f$ is Borel I know that $\{f \geq 0\}$ and $\{f < 0\}$ are measurable so I don’t have to worry about the integral being defined for these two pieces.
Because $|f| > M$ only on a set of measure zero, I can just set $f$ to $0$ at those points without changing anything, so I won’t make a note about that going forward.
This means
$$
\int f \dmu = \int f_+\dmu – \int f_-\dmu.
$$
For the first piece,
$$
0 \leq \int f_+\dmu \leq \int_{f \geq 0} M\dmu \leq M
$$
and for the second piece
$$
0 \geq -\int f_-\dmu \geq \int_{f < 0} -M\dmu > -\infty
$$
so $\int f\dmu$ is formed by adding two finite things and is therefore finite itself.
$\square$
Result 2: $\vp$ is always finite.
Pf: I can write $e^{itx}$ as
$$
e^{itx} = \cos(tx) + i\sin(tx)
$$
so
$$
\vp(t) = \int \cos(tx)\dmux + i\int\sin(tx)\dmux.
$$
Both $\sin$ and $\cos$ are bounded so by Result 1 $\vp$ is always finite.
$\square$
The main result I want to prove is this:
Theorem: if the $k$th absolute moment of $\mu$ is finite, then $\vp$ is differentiable $k$ times and
$$
\vp^{(k)}(t) = i^{k \text{ mod } 4}\int x^ke^{itx}\dmux.
$$
I’m also not worrying about just the $k$th absolute moment existing vs. the first $k$ absolute moments existing because the finiteness of $\int |x|^k\dmux$ implies finiteness for $0 \leq j < k$.
Before I prove this, I’ll prove a helper result that does most of the work. I’ll be using the Dominated Convergence Theorem (DCT) so I’ll state that here.
DCT: Let $(\Omega, \mathscr F, \nu)$ be a measure space and let $\{f_n : n \in \mathbb N\}$ be a collection of measurable functions from $\Omega$ to $\mathbb C$ that converge pointwise to a function $f_\infty$. Furthermore, suppose that there is some integrable function $g$ such that $|f_n(\omega)| \leq g(\omega)$ for all $\omega\in\Omega$ and $n \in \mathbb N$. Then
$$
\lim_{n\to\infty} \int f_n\,\text d\nu = \int f_\infty \,\text d\nu
$$
i.e. the limit and integral can be exchanged.
I’ll now move on to my result.
Result 3: If $(a_n)$ is any real-valued sequence that converges on $0$ (but never equals it) and $\int |x|^{n+1}\dmux < \infty$ then
$$
\lim_{n\to\infty} \int\left[\frac{e^{ixa_n} – 1}{a_n}\right]x^n e^{itx}\dmux = i \int x^{n+1}e^{itx}\dmux.
$$
Pf: this problem requires showing I can exchange the limit and integral so I’ll go about setting up the pieces for me to use the DCT. My measure space is $(\mathbb R, \mathbb B, \mu)$. I’ll take
$$
f_n(x) = \left[\frac{e^{ixa_n} – 1}{a_n}\right]x^n e^{itx}
$$
and note that $f_n$ converges pointwise to
$$
f_\infty(x) = ix^{n+1}e^{itx}
$$
($t$ is fixed but arbitrary here so I’m not including it as an argument for simplicity). These are all measurable so I’m good to go there.
The remaining, and hardest, piece is $g$. I need a function $g$ such that
$$
g(x) \geq |f_n(x)| \;\;\forall n\in\mathbb N, \;\forall x\in\mathbb R.
$$
I have
$$
|f_n(x)| = \left\vert\frac{e^{ixa_n} – 1}{a_n}\right\vert |x|^n
$$
since $|e^{itx}| = 1$. I know $|x|^n$ is integrable by assumption ($|x|^{n+1}$ being integrable implies that) so I’m just going to work on bounding $\left\vert\frac{e^{ixa_n} – 1}{a_n}\right\vert$ and see what happens.
$$
\left\vert\frac{e^{ixa_n} – 1}{a_n}\right\vert = \left\vert\frac{\cos(xa_n) – 1 + i\sin(xa_n)}{a_n}\right\vert \\
\leq \left\vert\frac{\cos(xa_n) – 1}{a_n}\right\vert + \left\vert\frac{\sin(xa_n)}{a_n}\right\vert
$$
so I’ll try to bound each of these pieces.
I did this with something of a guess and check where I plotted the pieces and came up with a guess that turned out to work: I claim each piece is bounded by $|x|$ no matter $n$.
Subclaim 1:
$$
\left\vert\frac{\cos(x\theta) – 1}{\theta}\right\vert \leq |x|
$$
for all $\theta\in\mathbb R\backslash \{0\}$.
Pf: Consider $f(y) = y^2 – (\cos y – 1)^2$ and note that this is a difference of squares so
$$
f(y) = (y – \cos y + 1)(y + \cos y – 1).
$$
I’ll let $f_+(y) := y + \cos y – 1$ and $f_-(y) := y – \cos y +1$ so $f = f_- \cdot f_+$. I’ll show that $\text{sgn}(f_-) = \text{sgn}(f_+)$ so the product $f$ is always non-negative.
In both cases, $f_-(0) = f_+(0) = 0$ so both functions have a root at $0$. Additionally,
$$
f’_-(y) = 1 + \sin y \geq 0\\
f’_+(y) = 1 – \sin y \geq 0
$$
so $f_-$ and $f_+$ are everywhere nondecreasing. Because $f’_-(0) = f’_+(0) = 1$ I know that $f_-$ and $f_+$ are increasing at $0$. I can use the increasing version of the lemma from this post to show that $f_-$ and $f_+$ have no other roots in some small neighborhoods of $0$. Then by monotonicity $0$ is each their only root so they always have the same sign. This proves that $f \geq 0$.
Rearranging $f$ and substituting $y = x\theta$ for some fixed $x \neq 0$ (which is a bijection so no issues there), I have
$$
y^2 \geq (\cos y – 1)^2 \\
\implies |y| \geq |\cos y – 1| \\
\implies |x\theta| \geq |\cos (x\theta) – 1| \\
\implies |x| \geq \left\vert\frac{\cos(x\theta) – 1}{\theta}\right\vert
$$
when $\theta \neq 0$. Also I can use the well-known limit
$$
\lim_{\theta\to 0} \frac{\cos(x\theta) – 1}{\theta} = x
$$
to confirm that there aren’t any issues at $\theta = 0$, even though this function has a hole there.
Subclaim 2:
$$
|x| \geq \left\vert\frac{\sin(x\theta)}{\theta}\right\vert
$$
for all $\theta\in\mathbb R\backslash \{0\}$.
Pf: This will follow an almost identical path as subclaim 1 so I’ll omit most of the details. I’ll consider the function
$$
y \mapsto y^2 – (\sin y)^2 = (y – \sin y)(y + \sin y)
$$
and can show that this is always non-negative, and then substituting in $x\theta= y$ finishes the claim. I again have a well-known limit
$$
\lim_{\theta\to 0} \frac{\sin(x\theta)}{\theta} = x
$$
as a justification for behavior near $\theta = 0$.
Returning to my proof, I’ve now shown that
$$
\left\vert\frac{e^{ixa_n} – 1}{a_n}\right\vert \leq \left\vert\frac{\cos(xa_n) – 1}{a_n}\right\vert + \left\vert\frac{\sin(xa_n)}{a_n}\right\vert \leq 2|x|
$$
for any $n\in\mathbb N$, so I can take $g(x) = 2|x|^{n+1}$ which by assumption is integrable, and then the DCT finishes off the proof.
$\square$
I can now prove the theorem itself.
Pf of theorem: by induction on $k$.
For the base case of $k=0$, I assume that $|x|^{k+1} = |x|$ is integrable and I want to show that
$$
\vp'(t) = i\int xe^{itx}\dmux.
$$
By definition,
$$
\vp'(t) = \lim_{h\to 0} \frac{\vp(t+h) – \vp(t)}{h} \\
= \lim_{h\to 0}\int \left[\frac{e^{ihx} – 1}{h}\right]e^{itx}\dmux.
$$
This limit exists if and only if it exists for every sequence $a_n \to 0$ with $a_n\neq 0$ for all $n\in\mathbb N$ so I can switch to an arbitrary sequence $(a_n)$ of that sort and consider
$$
\lim_{n\to\infty}\int \left[\frac{e^{ixa_n} – 1}{a_n}\right]e^{itx}\dmux.
$$
Result 3 did the heavy lifting of establishing that I can switch this limit and integral so
$$
\lim_{n\to\infty}\int \left[\frac{e^{ixa_n} – 1}{a_n}\right]e^{itx}\dmux \\
= \int\lim_{n\to\infty} \left[\frac{e^{ixa_n} – 1}{a_n}\right]e^{itx}\dmux \\
= i \int xe^{itx}\dmux
$$
as desired. This is independent of the path $(a_n)$ so it applies in general.
For the inductive case, I’ll assume true for $k=0,\dots,m$. Now I need
$$
\vp^{(m+1)}(t) = \frac{\text d }{\text dt} \vp^{(m)}(t) \\
= \lim_{h\to 0} \frac{\vp^{(m)}(t + h) – \vp^{(m)}(t)}{h}.
$$
By the inductive hypothesis this equals
$$
i^{m \text{ mod } 4}\lim_{h\to 0}\int \left[\frac{e^{ihx} – 1}{h}\right]x^me^{itx}\dmux
$$
and again by Result 3 this equals
$$
i^{m + 1 \text{ mod } 4}\int x^{m+1}e^{itx}\dmux
$$
as desired.
$\square$