By the end of the Civil War, Unionists from every southern state except South Carolina, where the war originated, sent regiments. How confusing it must have been to have to pick a side and go fight for it, even though your neighbor, cousin, or whatever fought for the other side. Some soldiers joined up, only to desert and enlist with the opposite side later on.
It seems that at the start of the war, sections in several southern states were in favor of preserving the Union, even though the majority didn't vote for Abraham Lincoln. Secession was a state decision, and wasn't put to the popular vote, so once the state joined the Confederacy, the men typically enlisted dutifully, in preservation of their own state. Slavery wasn't as issue, because most southern soldiers didn't own slaves, and those that did were often exempt from fighting. Some northern states allowed slavery, as well as several "border" states.
There were a few "fire eaters" who spewed venom, persuading others to join their crusade for the Confederacy. One such example is the following, written by a Kentucky rebel named James Blackburn, in a letter to his wife:
My Dear Wife: I have left you and our children in the land of the despot, but God grant that I may soon be able to make the union men of Kentucky feel the edge of my knife. From this day I hold every union traitor as my enemy, and from him I scorn to receive quarter and to him I will never grant my soul in death; for they are cowards and villains enough. Brother Henry and I arrived here without hindrance. I have had chills all the way, but I hope to kill 40 Yankees for every chill that I ever had.