28 November 2012

Rotation in 3D using OpenCV's warpPerspective

In order to easily rotate an image in 3D space, I have written a simple method that will do just that. It accepts rotations (in degrees) along each of the three axis (x, y and z), with 90 degrees being the "normal" position.
It also supports translations along each of the axis, and a variable focal distance (you would usually want the focal distance to be the same as your dz).

The parameters are:
input: the image that you want rotated.
output: the Mat object to put the resulting file in.
alpha: the rotation around the x axis
beta: the rotation around the y axis
gamma: the rotation around the z axis (basically a 2D rotation)
dx: translation along the x axis
dy: translation along the y axis
dz: translation along the z axis (distance to the image)
f: focal distance (distance between camera and image, a smaller number exaggerates the effect)

The method is defined as:



  void rotateImage(const Mat &input, Mat &output, double alpha, double beta, double gamma, double dx, double dy, double dz, double f)
  {
    alpha = (alpha - 90.)*CV_PI/180.;
    beta = (beta - 90.)*CV_PI/180.;
    gamma = (gamma - 90.)*CV_PI/180.;

    // get width and height for ease of use in matrices
    double w = (double)input.cols;
    double h = (double)input.rows;

    // Projection 2D -> 3D matrix
    Mat A1 = (Mat_<double>(4,3) <<
              1, 0, -w/2,
              0, 1, -h/2,
              0, 0,    0,
              0, 0,    1);

    // Rotation matrices around the X, Y, and Z axis
    Mat RX = (Mat_<double>(4, 4) <<
              1,          0,           0, 0,
              0, cos(alpha), -sin(alpha), 0,
              0, sin(alpha),  cos(alpha), 0,
              0,          0,           0, 1);

    Mat RY = (Mat_<double>(4, 4) <<
              cos(beta), 0, -sin(beta), 0,
              0, 1,          0, 0,
              sin(beta), 0,  cos(beta), 0,
              0, 0,          0, 1);

    Mat RZ = (Mat_<double>(4, 4) <<
              cos(gamma), -sin(gamma), 0, 0,
              sin(gamma),  cos(gamma), 0, 0,
              0,          0,           1, 0,
              0,          0,           0, 1);

    // Composed rotation matrix with (RX, RY, RZ)
    Mat R = RX * RY * RZ;

    // Translation matrix
    Mat T = (Mat_<double>(4, 4) <<
             1, 0, 0, dx,
             0, 1, 0, dy,
             0, 0, 1, dz,
             0, 0, 0, 1);

    // 3D -> 2D matrix
    Mat A2 = (Mat_<double>(3,4) <<
              f, 0, w/2, 0,
              0, f, h/2, 0,
              0, 0,   1, 0);

    // Final transformation matrix
    Mat trans = A2 * (T * (R * A1));

    // Apply matrix transformation
    warpPerspective(input, output, trans, input.size(), INTER_LANCZOS4);
  }
 
Example usage to rotate an image 45° around the y-axis:
rotateImage(orignalImage, outputImage, 90, 135, 90, 0, 0, 200, 200);

33 comments :

zousa said...

Thanks, just what I needed!

observe-ist said...

this is awesome. thanks mate

Unknown said...

Hi, Thanks for the excellent code. I tried with few inputs but couldnt get the desired results. Can you please provide some sample inputs, like rotating the image in y axis, z axis etc.,

Vicente said...

+1 Mathew.

I was trying it and there is something weird. With -90 and 90 in the axis looks like ok but if you are using 45 or 30 degrees or similiar the rotation is crazy.

is there any modification to do?

Michael Jepson said...

Hi Vicente,

How do you mean, the rotation is crazy?

Regards,
Michael

Vicente said...

Hello Michael.... sorry for my language.

I mean that i have doubts if the algorithm is ok.

If you put 45º or 50º in x-axis or any axis, actually the image looks like 10º. but if you put the normal values and reverse values:

rotateImage(src_img, dst_img, 90, 90, 90, 0, 0, 0, 1);
rotateImage(src_img, dst_img, 180, 90, 90, 0, 0, 0, 1);

it works ok. Weird.

Also, there is a very strange effect. If the output image is bigger than the source size you must to control it because, by default, if you pass the right border opencv draw (follow) in the left border and you get a "mirror" effect very odd.

I don't know... i only want to rotate a picture in the center with x,y,z axis in a loop.... but... may be the next time.

Vicente

Michael Jepson said...

Hi Vicente,

I think your f is too small. Try something like 200 or so. Play around with dz and f to get different results.

Regards,
Michael

Anonymous said...

The code works , warping is fine. But can you explain your code!

Unknown said...

Hi everyone!

This code works for me but I don't know you the Roll and Pitch angles are exchanged. When I change "alpha", the image is warped in pitch and when I change "beta" the image in warped in roll. Does someone know why it happens? Thanks

chela said...

Hi! Awesome work! I am trying to use your code. When rotating on the z axis it works fine, but on x and y I have to pass in extremely small values for it to work (I removed the radians calculation in your code and so the minus 90 degrees too) the rotations looks exaggerated (like, np.radians(0.01) for the y axis yields a pretty noticeable rotation, looks like 20 degrees to me. Maybe I am misunderstanding something or misusing dz and f values? I am setting a value of 5 there as the pic I am trying with states a focal length of 5 mm. I am setting dx=dy=0.
Also, the projection matrices confuse me a bit, Am I rotating around the center of the image or around a corner? I find it difficult to realize on x and y rotations.

Thanks!

chela said...

One more detail, if I change f and dz to 500 or 5000 instead of 5 starts yielding better results. In what units am I supposed to send those values? The pic says 5mm for focal length. Thanks! I am really excited I am seeing my pic rotate! :)

Michael Jepson said...

Hi Chelis,

dz = translation in z direction (translation as part of the whole operation)
f = focal distance of the camera, e.g. the distance between camera and picture. If this is too small, rotations will look exaggerated. I usually use f = 200 or 250. You'll usually want the same value for these, so you have approx. the whole image in view after the operation.
Rotation is always around the centre of the image, hence the translation of half the width and half the height in the 2D -> 3D and vice versa matrices.

Good to hear my code is of use to you!

Regards,
Michael

Unknown said...

Could you give me some reference of your method, such as some proofs, thank you.

Anonymous said...

A2 matrix is not correct. Last 2 columns are interchanges.

Unknown said...

Hello Micheal ,I am trying to rotate a 2d image with this code but the output is a blank image.what am i doing wrong?

Michael Jepson said...

Hi Shubham,

Did you use the example values for alpha, beta, gamma, dx, etc?
I know that if the image is at 90 degrees to the virtual camera (i.e. alpha or beta = 0), you won't see anything. Also, when the camera is too close (dz too small) it won't show either.
Please try my example values and see if they work for you. Then you can work from there. Please note that you have to pass 90 degree as alpha to make it appear straight towards the camera.

Best regards,
Michael

MrDaniel said...

Hello Michael,

This is awesome stuff!

However, when i try it with a non-square image i run into trouble.

There appears to be a problem with the aspect ratio.

Please see OpenCV forum for the details relating to the question.

http://answers.opencv.org/question/98301/mock-camera-intrinsics/

Regards,

Daniel

Unknown said...
This comment has been removed by the author.
Unknown said...
This comment has been removed by the author.
Unknown said...

mr jepson thank you veeery much for the excellent code,
could just pls say how could i change the origin ?

Alex P. said...

Great, thank you. For does who wants more explaination, they can read http://stackoverflow.com/questions/17087446/how-to-calculate-perspective-transform-for-opencv-from-rotation-angles

As pointed out already:
A2 matrix is not correct. Last 2 columns are interchanged.(to verify, with no rotation T should be eye)

Balakrishnan said...

Hi,

This is a very nice script, I have query on the Focal Length that you have used.
f = 200 meaning focal length is 200mm?

Anonymous said...

This is a great work. Is there any way to obtain plane normal vector from this formulas for each rotated image?

Elia said...

Thank you for the code. how can i change the background color from black (0,0,0) to a color i can send as a parameter to the function

Jack Ryan said...

Awesome post mate I loved it completely it's the dopest post I've ever seen in my entirety of existence on this planet.
Kilt

Liam Penny said...

Great post! Really loved reading it and it's written amazingly! Thanks for the cool share :) Appreciate it.
utility kilts

bdsaglam said...

Thanks for the post, really helped a lot. I think the last row of A2 matrix is wrong, the last element in the last row must be 1 and all the others zero.
Mat A2 = (Mat_(3,4) <<
f, 0, w/2, 0,
0, f, h/2, 0,
0, 0, 0, 1);

Michael Jepson said...

Hi bdsaglam, can you explain this?

bdsaglam said...

Well, I thought a projection matrix from 3D to 2D is this:

[1, 0, 0, 0,
0, 1, 0, 0,
0, 0, 0, 1]

so that when I multiply it with a vector [x, y, z, 1], it gives [x, y, 1]. Is there something I miss?

Michael Jepson said...

As far is I know, the last column needs to be all zeroes. It's been a while since I wrote this, so I'm not too familiar with the maths anymore.
This page: https://secomparteosepierde.blogspot.com/2018/03/3x4-projection-matrix.html seems to disagree with you though.
How are the results you're having with this change?

bdsaglam said...

Actually, I am very new to this subject so probably it's my mistake. I have just realized that when I tried to have a rotation around z axis with the code above, it did not give the same matrix with cv2.getRotationMatrix2D. Currently, I am implementing it in Python and got consistent matrices, but it does not look like how eye see. I think there is something about perspective transformation that I don't know yet. I'll definitely check out the article you gave. Thanks a lot.

erectile dysfunction drugs said...

Appreciating the time and energy you put into your site and in depth information you present. It's great to come across a blog every once in a while that isn't the same out of date rehashed material. Great read! I've saved your site and I'm including your RSS feeds to my Google account.

Marchingpony@1 said...

How to add thickness to a rotating image at particular angle.