For bosses that have a cooperative loot system, a minimum participation level is required in the fight to guarantee +1 kill on the bosstiary.
The amount of damage you deal is taken into account. In addition, taking damage from the boss and supporting other players during the fight (healing, for example) also helps ensure that you reach the minimum participation level.
Dying doesn't necessarily prevent you from getting the credit. However, going too long without participating in the fight can cause you to miss the credit.
For bosses that are not part of the cooperative loot system, a single hit is enough to secure the credit, provided the boss doesn't take too long to die.