Start by dividing the islanders into 3 groups of 4. Placing two groups results in two possible outcomes.

(1) If they balance, they are all real. That means one of the remaining is more or less.

Weigh 3 of the regular weighing guys against 3 of the remaining ones. If they balance, the last guy is more or less.

If it is lighter, one of the ones on the seesaw must be less. Weigh 2 of the suspicious ones. If they balance, then the third guy is less. If they don't balance, the lighter one is the guy we are looking for.

If it is heavier, one of the ones on the seesaw must be more. Weigh 2 of the suspicious ones. If they balance, then the third guy is more. If they don't balance, the heavier one is the guy we are looking for.

(2) If at first they don't balance, the remaining 4 are real. Replace 3 of the ones on the "heavier" side with 3 of the ones on the "lighter" side and add 3 real guys on the "lighter" side. If the same side is still heavier, it means that either the one that wasn't replaced on the "heavier" side is heavier or the one not replaced on the "lighter" side is lighter. Choose either of them, and weigh them against a real guy.

Also, if they balance at the second use, then one of the three "lighter" ones were fake. Weigh two against each other, If they balance, then the third guy is fake. If they don't balance, the lighter one is the guy we are looking for.