There's no denying that the Draft Riots in July 1863 had a strong component of racism. The white workers who rioted were said to be mostly Irish immigrants, though reports at the time said German immigrants and other ethnic groups also participated.
Poor whites in the north had become increasingly alienated from the war effort throughout 1862. And many felt that the Emancipation Proclamation, issued at the beginning of 1863, had been a betrayal. Lincoln's detractors said he expected the poor whites to fight and die to liberate blacks in the south, who then might move northward and take jobs held by white immigrants.
Even the governor of New York had given a speech on July 4th in which he said Lincoln had exceeded his authority by coercing men to fight a war on behalf of blacks.
No doubt incited by such rhetoric, the mobs roaming through the streets during the New York draft riots began to target African-Americans, thousands of whom had been living freely in New York City. Dozens of blacks were assaulted, tortured, and even doused with fuel and set afire. Some were lynched, and left hanging from tree branches.
One of the most notorious acts of violence occurred on Clarkson Street, on the West Side of Manhattan. A black workman named William Jones was hanged from a tree, and a bonfire was then set beneath his body.
This engraving of the horrific scene was published in the Illustrated London News.