28 November 2012

Rotation in 3D using OpenCV's warpPerspective

In order to easily rotate an image in 3D space, I have written a simple method that will do just that. It accepts rotations (in degrees) along each of the three axis (x, y and z), with 90 degrees being the "normal" position.
It also supports translations along each of the axis, and a variable focal distance (you would usually want the focal distance to be the same as your dz).

The parameters are:
input: the image that you want rotated.
output: the Mat object to put the resulting file in.
alpha: the rotation around the x axis
beta: the rotation around the y axis
gamma: the rotation around the z axis (basically a 2D rotation)
dx: translation along the x axis
dy: translation along the y axis
dz: translation along the z axis (distance to the image)
f: focal distance (distance between camera and image, a smaller number exaggerates the effect)

The method is defined as:



  void rotateImage(const Mat &input, Mat &output, double alpha, double beta, double gamma, double dx, double dy, double dz, double f)
  {
    alpha = (alpha - 90.)*CV_PI/180.;
    beta = (beta - 90.)*CV_PI/180.;
    gamma = (gamma - 90.)*CV_PI/180.;

    // get width and height for ease of use in matrices
    double w = (double)input.cols;
    double h = (double)input.rows;

    // Projection 2D -> 3D matrix
    Mat A1 = (Mat_<double>(4,3) <<
              1, 0, -w/2,
              0, 1, -h/2,
              0, 0,    0,
              0, 0,    1);

    // Rotation matrices around the X, Y, and Z axis
    Mat RX = (Mat_<double>(4, 4) <<
              1,          0,           0, 0,
              0, cos(alpha), -sin(alpha), 0,
              0, sin(alpha),  cos(alpha), 0,
              0,          0,           0, 1);

    Mat RY = (Mat_<double>(4, 4) <<
              cos(beta), 0, -sin(beta), 0,
              0, 1,          0, 0,
              sin(beta), 0,  cos(beta), 0,
              0, 0,          0, 1);

    Mat RZ = (Mat_<double>(4, 4) <<
              cos(gamma), -sin(gamma), 0, 0,
              sin(gamma),  cos(gamma), 0, 0,
              0,          0,           1, 0,
              0,          0,           0, 1);

    // Composed rotation matrix with (RX, RY, RZ)
    Mat R = RX * RY * RZ;

    // Translation matrix
    Mat T = (Mat_<double>(4, 4) <<
             1, 0, 0, dx,
             0, 1, 0, dy,
             0, 0, 1, dz,
             0, 0, 0, 1);

    // 3D -> 2D matrix
    Mat A2 = (Mat_<double>(3,4) <<
              f, 0, w/2, 0,
              0, f, h/2, 0,
              0, 0,   1, 0);

    // Final transformation matrix
    Mat trans = A2 * (T * (R * A1));

    // Apply matrix transformation
    warpPerspective(input, output, trans, input.size(), INTER_LANCZOS4);
  }
 
Example usage to rotate an image 45° around the y-axis:
rotateImage(orignalImage, outputImage, 90, 135, 90, 0, 0, 200, 200);

17 comments :

zousa said...

Thanks, just what I needed!

observe-ist said...

this is awesome. thanks mate

Mathew iprocessor said...

Hi, Thanks for the excellent code. I tried with few inputs but couldnt get the desired results. Can you please provide some sample inputs, like rotating the image in y axis, z axis etc.,

Vicente said...

+1 Mathew.

I was trying it and there is something weird. With -90 and 90 in the axis looks like ok but if you are using 45 or 30 degrees or similiar the rotation is crazy.

is there any modification to do?

Michael Jepson said...

Hi Vicente,

How do you mean, the rotation is crazy?

Regards,
Michael

Vicente said...

Hello Michael.... sorry for my language.

I mean that i have doubts if the algorithm is ok.

If you put 45º or 50º in x-axis or any axis, actually the image looks like 10º. but if you put the normal values and reverse values:

rotateImage(src_img, dst_img, 90, 90, 90, 0, 0, 0, 1);
rotateImage(src_img, dst_img, 180, 90, 90, 0, 0, 0, 1);

it works ok. Weird.

Also, there is a very strange effect. If the output image is bigger than the source size you must to control it because, by default, if you pass the right border opencv draw (follow) in the left border and you get a "mirror" effect very odd.

I don't know... i only want to rotate a picture in the center with x,y,z axis in a loop.... but... may be the next time.

Vicente

Michael Jepson said...

Hi Vicente,

I think your f is too small. Try something like 200 or so. Play around with dz and f to get different results.

Regards,
Michael

Anonymous said...

The code works , warping is fine. But can you explain your code!

Matheus Laranjeira said...

Hi everyone!

This code works for me but I don't know you the Roll and Pitch angles are exchanged. When I change "alpha", the image is warped in pitch and when I change "beta" the image in warped in roll. Does someone know why it happens? Thanks

Chelis said...

Hi! Awesome work! I am trying to use your code. When rotating on the z axis it works fine, but on x and y I have to pass in extremely small values for it to work (I removed the radians calculation in your code and so the minus 90 degrees too) the rotations looks exaggerated (like, np.radians(0.01) for the y axis yields a pretty noticeable rotation, looks like 20 degrees to me. Maybe I am misunderstanding something or misusing dz and f values? I am setting a value of 5 there as the pic I am trying with states a focal length of 5 mm. I am setting dx=dy=0.
Also, the projection matrices confuse me a bit, Am I rotating around the center of the image or around a corner? I find it difficult to realize on x and y rotations.

Thanks!

Chelis said...

One more detail, if I change f and dz to 500 or 5000 instead of 5 starts yielding better results. In what units am I supposed to send those values? The pic says 5mm for focal length. Thanks! I am really excited I am seeing my pic rotate! :)

Michael Jepson said...

Hi Chelis,

dz = translation in z direction (translation as part of the whole operation)
f = focal distance of the camera, e.g. the distance between camera and picture. If this is too small, rotations will look exaggerated. I usually use f = 200 or 250. You'll usually want the same value for these, so you have approx. the whole image in view after the operation.
Rotation is always around the centre of the image, hence the translation of half the width and half the height in the 2D -> 3D and vice versa matrices.

Good to hear my code is of use to you!

Regards,
Michael

Zheqi He said...

Could you give me some reference of your method, such as some proofs, thank you.

Anonymous said...

A2 matrix is not correct. Last 2 columns are interchanges.

Shubham Jaiswal said...

Hello Micheal ,I am trying to rotate a 2d image with this code but the output is a blank image.what am i doing wrong?

Michael Jepson said...

Hi Shubham,

Did you use the example values for alpha, beta, gamma, dx, etc?
I know that if the image is at 90 degrees to the virtual camera (i.e. alpha or beta = 0), you won't see anything. Also, when the camera is too close (dz too small) it won't show either.
Please try my example values and see if they work for you. Then you can work from there. Please note that you have to pass 90 degree as alpha to make it appear straight towards the camera.

Best regards,
Michael

MrDaniel said...

Hello Michael,

This is awesome stuff!

However, when i try it with a non-square image i run into trouble.

There appears to be a problem with the aspect ratio.

Please see OpenCV forum for the details relating to the question.

http://answers.opencv.org/question/98301/mock-camera-intrinsics/

Regards,

Daniel