The driver cannot suspend before it has loaded completely. So the test
for device pointers is not required. Remove it.

v3:
	* s/them/it/ in commit message

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20200617080340.29584-3-tzimmermann@suse.de