The plant is actually mildly toxic all over but the highest concentrations of the toxic compounds are found in the rhizomes (root-like stems). These compounds have traditionally been used in herbalism but western science has also transformed them into useful cancer-killing drugs.
Despite its toxicity the mayapple fruit is used to make jams and jellies. Deliciously irritating!
Sources: Kaczmarek, Frank (2009). New England Wildflowers. Guilford, CT: Falcon Guides.
Illinois Wildflowers: http://www.illinoiswildflowers.info/woodland/plants/mayapple.htm
Alternative Nature Online Herbal: http://www.altnature.com/gallery/mandrake.htm