with a set of values defining Triangles I need to know weather a point is in or out of the triangle. Also, for any point that is in the triangle, the code must check if it is in the edge or not.
The following strategy seams to make some sence for the Point in triangle edge situation, however it may not work properly due to float to int roundings.
* Get the two points that define the edge
* Use the y = m x + b and m = y2 - y1 / x2 - x1 to find 'b' and 'm'
* With the coordinates for the PointNew see if it's in the line and between the other two points. If so, PointNew is in the edge.
And this algorithm would be applied after the coordinates are transformed into perspective projection.
Do you think this will give good (or at least decent) results?
Any other proven techniques?
BTW: Working based only on software rendering, not using DX or Opengl.
Thanks for your comments. m .