What may be the case here, is that you're checking on records on which the checkbox hasn't been checked/unchecked before. The value of that checkbox is NULL.
So, if you're condition is like: checkbox does not equal No, the condition will evaluate to "true" when the checkbox is either Yes or NULLSee below, this is a Pipeline checkbox on one of my forms. Looks like its value is No....
But when you look at the actual field value (with the LevelUp tool):
If you've found this thread useful, dive deeper into User Group community content by role