I've tried 3, 6, 9, and 12 and none of them worked xd

I gave up on the problem and apparently the answer is 11, thanks anyways Melody and Rom.

Solution:

We can construct segments of length 1,sqrt2 ,sqrt5 ,sqrt10 ,2 ,2sqrt2 ,sqrt13 ,3 ,3sqrt2 (these can be obtained systematically by considering all lengths of segments from the top left dot to other dots). The following results can be obtained either by inspection or analysis of slope.

For each of 1, sqrt5, sqrt13, 3, there are zero isosceles triangles having it as its base length.

For sqrt10, there is one.

For each of 2sqrt2, 3sqrt2, there are two.

For each of 2, sqrt2, there are three.

This gives a total of 1 + 2 + 2 +3 + 3 = 11 non-congruent isosceles triangles.