The present study investigates the temporal dynamics of the brain activity predicting the sensory outcomes of observed hand-object interactions of others. Participants are presented with pictures of a hand grasping or withdrawing from noxious and neutral objects. They are then asked to judge whether this hand-object interaction causes painful consequences. In the early stages of stimulus processing, the effect of action was observed in the event-related potential components N1 and N2. Significant interactions of action * object were observed in the later components P3 and late positive potential (LPP): only when the object was noxious, the action "grasp" elicited a significantly larger amplitude than the action "withdrawal". These results suggest that: on the one hand, when observing the hand-object interaction from the third-person perspective, the action type of others can be processed in an automatic style. On the other hand, integrating the information of action and object to predict the sensory consequence of this interaction is a top-down, cognitive controlled processing. The current findings are different from previous studies using first-person perspective visual stimuli which support that the processing of hand-object interaction is rapid and automatic.