Hi William,
The algorithm draws a lot of triangles with players at the vertices. I appears that the final result for P is the polygon enclosed by the red boundary in the code below (the convex hull). After that, what the last line is doing is not so clear. Are you just looking for the area inside the boundary? If it's no more complicated than that, then the following code should get it done.
Let the players be at locations (x,y) and let vx be the vector of x coordinates and vy be the vector of corresponding y coordinates. Then
% make up some data vx = 100*rand(1,11); vy = 50*rand(1,11);
ind = convhull(vx,vy) % indices of players on the boundary plot(vx,vy,'o',vx(ind),vy(ind)); xlim([0 100]); ylim([0 50]); A = polyarea(vx(ind),vy(ind))