When was the symbol "U" first used to denote voltage or electric potential difference?

In my opinion, the etymological accounts for the letter $U$ from Latin are reverse histories. People thought to rationalize a symbol with the best anecdotal evidence. Sometimes, there is no story behind a symbol just like p of the $p$H, just like L for the momentum, or even sinc function (nobody has a definite proof where $c$ came from. Sometimes the notation could be just alphabetical order. My German is quite limited but machine translations have made life much easier for science historians see for example Facile Solutions to the Problems Associated with Chemical Information and Mathematical Symbolism While Using Machine Translation Tools Paper

$U$, $V$, and $W$ are all over 19th-century electricity books for potential or related ideas. One can see the alphabetical order closeness. Thanks to Georg who found this history book. You already found that $U$ was everywhere for potentials in mechanics.

First example: Helmholtz, Hermann von Helmholtz: Über die Erhaltung der Kraft eine physikalische Abhandlung (1847). Book

Through all methods of electrification equal quantities of positive and negative electricity are produced; in the equalization of the electricities between two bodies, of which one, $A$, contains just as much positive electricity as the other, $B$, negative, one half of the positive electricity passes from $A$ to $B$, whereas one half of the negative passes from $B$ to $A$. If we call the potentials of the bodies with respect to themselves $W_a$ and $W_b$, and their potential with respect to one another $V$, then we obtain the total kinetic energy gained when we subtract, from these same potentials after the motion, the potential of the transferring electrical masses before the motion with respect to each of the other masses and to themselves.

Then come to the Hoppe, E. (1884). Geschichte der Elektrizität. Barth.Book.

On page 258.

Let $U$ denote the density of the electricity at a point (the tension=Spannung), $k$ the reciprocal value of the resistance characteristic of the conductor, i.e. the resistance of a conductor of length ($l$) and cross–section 1, $q$ the cross–section of the wire, and $N$ the direction of the wire. Then in the time 1 an amount of electricity

$$ e = k q \frac{\partial U}{\partial N} $$

flows through the cross–section. But $\dfrac{\partial U}{\partial N}$ is nothing other than the potential gradient, so

$$ e = k q \frac{a}{l},$$

or, if one sets $\dfrac{l}{k q} = w$ for the resistance,

On page 364.

1. In addition to Dirichlet’s lectures, compare especially his paper Sur un moyen général etc. in Crelle’s Journal, vol. 32, where he proves purely analytically the theorem, assumed without proof by Green, that there exists one and only one function ($U$) in Green’s sense, which had been proved by Gauss himself from the principles of potential theory. The characteristic properties of a surface potential are also to be found in the Monatsberichte of the Berlin Academy, 1846, p. 211.

on page 536:

The complete development of this principle was left to a later time. In the course of development, the law of conservation of force was then transformed into the law of conservation of energy, namely that, when a system of points is moved under the action of external forces, for each time element $$ d(T + U + U^{0} - V) = dS $$ where $T$ is the kinetic energy, $U^{0}$ the potential of the system in the mechanical sense, $U$ the electrostatic potential, $V$ the electrodynamic potential, and $S$ the work expended; that is, the increase of the energy of the system is equal to the work expended during one and the same time element.

So all it seems, it was just $U$, $V$ and $W$, the alphabetical ordered choices, now in the last equation $T$ and $S$ are included!

• Hoppe, E. (1884). Geschichte der Elektrizität. Barth.

Hoppe confirms ACR's suggestion that U found its way from Laplace into German electricity literature. The passage starts at p. 338 and specifically note the footnote 1) on that page.

Apparently it was Kirchhoff who in his work, using Laplace/Gauss/Green potential theory adopted the letter u for potential (p.338-345).

While the phrase "Potentialunterschied" is tempting, I found that the literature often used "Spannungsdifferenz" (see for example Hoppe p. 344) or "Potentialsdifferenz" (for an example see below).

Compare:

• Kirchhoff, S. (1845). Ueber den Durchgang eines elektrischen Stromes durch eine Ebene, insbesondere durch eine kreisförmige. Annalen der Physik, 140(4), 497-514.

For example in:

• Neumann, F. E. (1848). Ueber ein allgemeines Princip der mathemat. Theorie inducirter elektrischer Ströme: Vorgelesen in der Berliner Akademie der Wissensch. am 9 Aug. 1847. G. Reimer.

Neumann uses the letters to P and Q to indicate potential. Neumann has no explicit letter for potential difference, and always spells it out. Kirchhoff was studying with Neumann, and Neumann had already adopted Kirchhoff's work (see p. 5), but he had not adopted his choices of symbols. Neumann would say "Potentials-Differenz" (compare p.28).