Out of the box, Tap to Wake is enabled on the iPhone X. Meaning, when the screen is off, a simple tap anywhere on the screen will wake it and instantly trigger Face ID.
If this is something that annoys you, disable it by going to Settings > General > Accessibility> Tap to Wake.
With Tap to Wake disabled, you will need to pick up the phone or press the side button to wake it.