There are many, but I will highlight a few hereunder.
They have a different combination of gonosomes - DNA make up of the 23rd pair of chromosomes in each cell - these genes determine primary and secondary sexual characteristics as well as some other features and also sometimes carry sex-linked characteristics.
During cell division by meiosis, these form in the respective genders.
During sexual intercourse, the female releases both X genes and the male releases either an X or a Y gene and hence the male determines the sex of the baby and what its gonosome combination will be.
Sometimes gene mutations can also occur resulting in XXX, XXY, XYY. This may lead to altered sexual preferences in each gender.
In addition offcourse the male and female reproductive organs are different in terms of males having penis, testes, etc as opposed to female's vagina, uterus, etc. Also the endocrine systems are different with males have lots of testosterone hormone which plays roles in secondary male characteristics such as body hair, deep voice, muscular bodies, etc , as opposed to females having lots of oestrogen and progesterone hormones which play important roles in the menstrual cycle, as well as breast structures, etc.