It’s difficult to determine, based on the picture (any picture), if a darker or brighter portion of a brick is mold contaminated or not. Assuming that it is, it would be impossible without a laboratory testing to issue any toxicity statements.
Although I’m not 100% sure but it appears that exposed brick sections are within the common wall between the two row homes, not the exposed rear extension of the house. That means that there’s another unit on the other side of that brick, so no moisture to feed or cause mold growth. If this is the case I’d say that discoloration might be a “side-effect” of the brick manufacturing process (sometimes created on purpose) or imperfection residue resulting from sandblasting / cleaning after re-pointing.
You can ask your home inspector to look for any moisture presence on those exposed bricks as well as on the interior finished wall areas corresponding to peeling paint on outside walls (below the widows and behind the AC unit (just in case…).
You may want to ask the inspector to do the same (moisture testing) for the fireplace bricks and definitely check the exposed above roof sections of both chimneys (the rear, visible on pictures, is probably for the heating system and there should be another one towards the front.
This should be done to make sure that both chimneys are in good condition (the rear one doesn’t look too good but it might be just the picture angle), their crowns are intact, and flues have functional caps (and other things your inspector should evaluate
Please let me know if you have any additional questions, just click “comment” below. Good luck on your transaction!
PS. Can't say much about that smell... empty homes often have that.