Go to any amateur bike race and look around at the riders. How many of them are hunched over in the upper back? The technical term for this spinal alignment is thoracic kyphosis. Some causes of thoracic kyphosis include tight pectoral muscles, tight lower back and glutes, or a tight neck. Cyclists tend to get upper spine issues because arching your upper spine is a great way to make up space needed to get to the handlebars. Thinking of a rider from the side, there is a finite length from the saddle to the handlebars. This space can be taken up by the rider horizontally in the traditional time trial position, or more commonly, the rider has a more relaxed upper body angle and reaches down with their arms to meet the hoods/drops. In the event of a tight posterior chain (read hamstrings and glutes), a rider may posteriorly rotate their hips to allow their hamstrings and glutes to relax. This rotation makes the effective hip angle tighter, forcing the rider to sit up more at their lower back. In order to make up the vertical space, the upper spine excessively curves downward toward the handlebars.
Chronic positioning like this maintains a closed chest (shortened pecs), but also lengthens both rhomboids and latissimus dorsi. Where is the problem here? Well, the rhomboids should be flexed during riding in order to protect the neck from excessive strain and keep the upper back flat across the shoulders. Chronic neck and upper back pain comes from a constantly arched back. Additionally, if a rider can sit up straighter, latissimus dorsi engagement can significantly improve torso stiffness. This stiffness can help with power transfer during time trials where leverage can be gained from pulling up on the skis and during road cycling when pulling up on the drops for leverage in a sprint or pulling on the hoods for leverage during a supramaximal effort.
In addition to lower pain and better power transfer during high intensity efforts, a large curvature in one’s upper spine can indicate issues in other parts of the body. If the cause of the excessive curvature is initially set on by a tight posterior chain causing anterior pelvic tilt, perhaps this curvature is the hint you needed that there are other underlying issues as well related to the posterior chain.
A door frame stretch to length the pecs and weight training to strengthen the lats and rhomboids are good local solutions. Specifically for the stretch, use a single bent arm and place it to the side of a door frame. Lean your body through the frame while holding the arm in place, you should feel a stretch across the proximal side of your chest.
Rhomboids can be strengthened by pulling your shoulder blades together and opening your chest. The best way to do this is with free weights. Lay prone face down on a bench with weights in each hand (10-15lbs) and squeeze your shoulder blades together and pull your elbows upwards toward the sky. Don't extend the elbows, keep them bent. We don't want extra muscle hypertrophy in the triceps. Another version of this exercise is commonly referred to as 'bent-over rows' although I don't recommend that exercise as the form is more difficult to get right and it's common to go into lumbar extension. When you are done, it should feel like something invisible is pulling your shoulders backward and lifting your chest and you should feel a bit like a military grunt.
For lat strengthening, do pull ups. Most cyclists can't do many or any pull ups but they are important for good power transfer. Caleb Ewan does a lot of pull ups, you can tell by the way his back musculature and it helps him control the bike during his sprints and get good core engagement. If you can't manage to do a pull up, use an assisted machine, with a focus on maximal contractions. Pick an assistance weight that allows you to do only 3 reps. Try to get 3 sets of this in and you'll quickly find yourself having the ability to do unassisted pull ups.
Even if you fix these local areas, the big problem is that you hunch your back because you have excessive posterior tilt in your pelvis. Other recommendations include stretching your hamstrings and adductor magnus (runners lunge). As well as trying to get better glute function and length.