Is it grammatically correct to use Neither in the start of a sentence? Example- Neither would the toddler give up the toy nor come inside.
To my knowledge, it is grammatically correct to use neither at the start of a sentence, but not like that.
Neither doesn't really work at the start of that sentence, I think it would have to be "The toddler would neither give up the toy nor come inside". But you could use neither at the start if it were "Neither toddler would give up the toy." as in there are multiple toddlers.
It's fine to use Neither...nor to introduce a sentence, but your example is awkward.
There's no rule that says the conjunction "neither...nor" should not be used to introduce a clause. Your example, however, is awkward and would be better stated in another way.
An example of good usage in the beginning of a sentence might be:
Neither I nor my husband wanted to cook dinner.
While it's not incorrect, I think your sentence has "neither" and "nor" too far apart to sound good. Put them closer together, and it makes a huge difference:
The toddler would neither give up his toy nor come inside.
The toddler would neither give up the toy nor come inside.
Most important issue here is to have the subject of the sentence in mind.
The focus should be on what specifically the toddler is giving up. Once neither is placed at the beginning, the sentence leads the reader to come to a different interpretation than what you intend. This is known as a garden path sentence.