Picking Objects on Multiple Marker

Jun 10, 2012 at 8:36 AM
Edited Jun 10, 2012 at 8:42 AM

Hello,

Currently, I am creating a simple application that projects different objects above different markers and users can read the data registered on GeometryNode's userdata using mouse.

Previously, I have successfully do the picking objects thing by following the tutorial shown on Tutorial 8 on single marker with multiple objects and successfully recognize the userdata registered on each geometry objects.

but the problem came when I separate each objects to different markers . The assigned 3D objects on the marker is shown on the screen and I can see the userdata too but somehow it still picked objects and userdata on different markers! (and the weird thing is my mouse is pointing at a space where there is no objects on it!)

Firstly, I assumed that this was a problem in ray picking algorithm, so I changed it to use only the marker that is detected by camera but it still detects the object on different markers 

here is the part of code I use to do ray picking

 

 foreach (MarkerNode marker in registeredMarkers)
	{
		if (marker.MarkerFound)
		{
				Vector3 nearSource = new Vector3(mouseLocation.X, mouseLocation.Y, 0);
				Vector3 farSource = new Vector3(mouseLocation.X, mouseLocation.Y, 1);

 // ray pick on each marker's view matrix
				Matrix viewMatrix = marker.WorldTransformation * State.ViewMatrix;
				
				Vector3 nearPoint = SimpleApp.GraphicsDevice.Viewport.Unproject(nearSource,
						State.ProjectionMatrix, viewMatrix, Matrix.Identity);
				Vector3 farPoint = SimpleApp.GraphicsDevice.Viewport.Unproject(farSource,
						State.ProjectionMatrix, viewMatrix, Matrix.Identity);

				List<PickedObject> pickedObjects = ((NewtonPhysics)SimpleApp.MainScene.PhysicsEngine).PickRayCast(
						nearPoint, farPoint);
			
			// etc
			// etc
			
		}
	}

Did I just do something wrong? If so please tell me where I made a mistake so that I can change the code. 

and my scene graph is pretty simple too something like this :

before separating :

RootNode -> MarkerNode -> TransformNode -> GeometryNode

   -> TransformNode -> GeometryNode

   -> TransformNode -> GeometryNode

   -> etc...

after separating :

RootNode -> MarkerNode -> TransformNode -> GeometryNode

-> MarkerNode -> TransformNode -> GeometryNode

-> MarkerNode -> TransformNode -> GeometryNode

-> etc...

any Idea?

Coordinator
Jun 10, 2012 at 9:14 AM

Please read this post: http://goblinxna.codeplex.com/discussions/207594

Jun 10, 2012 at 2:42 PM

Thank you for your fast reply ,

I've read the post from the link you have provided and am i correct whenever the camera detects a marker, a whole physics objects in physics engine will be mapped on that marker even though there are 3d objects that are not present (objects that attached to another marker but registered in physics engine) ? so if the camera detects more than one marker, every marker has the same physics objects? please correct me if I am wrong. 

I also have tried to unregister the physics (set addToPhysicsEngine to false ) from physics engine when the marker is not detected and register it when marker is detected. its working right when the camera captures single marker, but not when capturing multiple marker.

Actually what I may need is know how to ray picking on different markers, these past five hours, I tried to do the triangle picking but still stuck

any other solution that I can try?

 

Thank you for this great work of Goblin XNA :)  

Coordinator
Jun 11, 2012 at 3:43 AM

Physics in AR is quite tricky. It's currently designed to work well with physics simulation in AR rather than simple picking, so you will need to work around a bit to make the picking happen correctly. If you see Tutorial 8 that for simple collision detection, you need to manually "transport" the object in the physics engine by using the NewtonPhysics.SetTransform method. Change of marker transformation won't affect the transformation stored in the physics engine to change because that will mess up a lot of stuff when you perform actual physical simulation. So if you want to force the transformation in the physics engine to change based on the marker transform, you need to manually do it through SetTransform method (specific to NetwonPhysics)

Ohan

Jun 12, 2012 at 9:28 AM

Hello eng_fron, few days ago I had the same problem that you, now i have a function that i think works well. I paste to you this function down. In my function I do more things that only picking, but maybe you can adapt it for your own proposes.

Sorry for my bad english :(

 

static public void Pick(MouseEventArgs e)
        {
            Dictionary<int, PickedObject> findedList = new Dictionary<int, PickedObject>();


            if (mouseDown)
            {
                mouseDown = false;

                selectedVisualization.Box.Enabled = false;
                selectedVisualization = null;


                foreach (KeyValuePair<int, GeometryNode> pair in NodeRegister.planeNodes)
                {
                    pair.Value.Physics.Pickable = false;
                    pair.Value.Enabled = false;
                    pair.Value.AddToPhysicsEngine = false;
                }



                markerToTransform = -1;
                return;
            }

            if (e.Button == MouseButtons.Left)
            {
                Vector3 nearSource = new Vector3(e.X, e.Y, 0);
                Vector3 farSource = new Vector3(e.X, e.Y, 1);

                // Debemos hacer una aproximaci�n de los intervalos:
                nearSource.X = nearSource.X * 1024 / form.XnaPanel.Width; 
                nearSource.Y = nearSource.Y * 768 / form.XnaPanel.Height;


                farSource.X = farSource.X * 1024 / form.XnaPanel.Width;
                farSource.Y = farSource.Y * 768 / form.XnaPanel.Height;



                // Debemos analizar cada marcador para comprobar el pick
                for (int i = 0; i < 6; i++)
                {
                    // Primero comprobamos que hemos localizado el marcador y que tiene un modelo asociado
                    if (MarkerController.Markers[i].MarkerFound && MarkerController.Markers[i].Children.Count > 0)
                    {
                        // En caso de que se den las condiciones calculamos la viewMatrix para este marcador
                        Matrix viewMatrix = MarkerController.Markers[i].WorldTransformation * GoblinXNA.State.ViewMatrix;

                        // Calculamos los puntos necesarios para el RayCasting
                        Vector3 nearPoint = sceneController.GraphicsDevice.Viewport.Unproject(nearSource, GoblinXNA.State.ProjectionMatrix, viewMatrix, Matrix.Identity);
                        Vector3 farPoint = sceneController.GraphicsDevice.Viewport.Unproject(farSource, GoblinXNA.State.ProjectionMatrix, viewMatrix, Matrix.Identity);

                        ((NewtonPhysics)sceneController.Scene.PhysicsEngine).RestartsSimulation();

                        // Ahora calculamos el RayCasting y obtenemos los objetos interceptados
                        List<PickedObject> pickedObjects = ((NewtonPhysics)sceneController.Scene.PhysicsEngine).PickRayCast(nearPoint, farPoint);


                        // Si hemos interceptado alg�n objeto
                        if(pickedObjects.Count > 0)
                        {
                            // Ahora debemos mirar si en los picked se encuentra el modelo asociado al marker
                            List<GeometryNode> pickedNodes = new List<GeometryNode>();

                            TransformNode associated = (TransformNode)MarkerController.Markers[i].Children[0];
                            GeometryNode geometryAssociated = (GeometryNode)(associated.Children[0]);


                            PickedObject nodeAssociated = null;

                            foreach (PickedObject p in pickedObjects)
                            {
                                string a = ((GeometryNode)p.PickedPhysicsObject.Container).Name;
                                if (geometryAssociated == ((GeometryNode)p.PickedPhysicsObject.Container))
                                {
                                    nodeAssociated = p;
                                }
                            }

                            // Ahora ya sabemos si hemos picado en el objeto asociado al marcador, y si lo hemos hecho lo guardamos para poder comparar despu�s
                            if (nodeAssociated != null)
                                findedList.Add(i, nodeAssociated);
                        }
                    }
                }


                // Cuanto terminamos el bucle tenemos todos los pickedObject que hemos atravesado y que est�n asociados con sus respectivos marcadores
                // Ahora debemos comparar cu�l est� mas cerca


                int selectedMarker = -1;
                bool firstFinded = false;


                for (int i = 0; i < 6; i++)
                {
                    if (findedList.ContainsKey(i))
                    {
                        if (!firstFinded)
                        {
                            selectedMarker = i;
                            firstFinded = true;
                        }
                        else if (findedList[i].IntersectParam < findedList[selectedMarker].IntersectParam)
                        {
                            selectedMarker = i;

                            // Esto lo ponemos para que quede constancia del marcador, as� despu�s podremos buscar el plano f�cilmente
                            
                        }
                    }
                }

                markerToTransform = selectedMarker;

                // Ahora ya sabemos cu�l es el seleccionado, as� que marcamos la caja (si es que hay alg�n seleccionado)

                if (selectedMarker != -1)
                {
                    TransformNode t = (TransformNode)MarkerController.Markers[selectedMarker].Children[0];
                    GeometryNode g = (GeometryNode)t.Children[0];

                    // Obtenemos la visualizaci�n asociada

                    Visualization v = NodeRegister.GetVisualization(g);

                    if (selectedVisualization != null)
                        selectedVisualization.Box.Enabled = false;

                    v.Box.Enabled = true;

                    selectedVisualization = v;


                    // Adem�s vamos a hacer que el �nico plano pickable sea el del objeto seleccionado, de esta forma no puede haber fallos
                    // al encontrar m�s de un plano 

                    // Hacemos que todos los planos dejen de ser pickables
                    foreach (KeyValuePair<int, GeometryNode> pair in NodeRegister.planeNodes)
                    {
                        pair.Value.Physics.Pickable = false;
                        pair.Value.Enabled = false;
                        pair.Value.AddToPhysicsEngine = false;
                    }


                    // Ahora hacemos pickable el del marker seleccionado
                    NodeRegister.planeNodes[selectedMarker].Physics.Pickable = true;
                    NodeRegister.planeNodes[selectedMarker].Enabled = true;
                    NodeRegister.planeNodes[selectedMarker].AddToPhysicsEngine = true;

                }
                else
                {
                    if (selectedVisualization != null)
                    {
                        selectedVisualization.Box.Enabled = false;
                        selectedVisualization = null;

                        markerToTransform = -1;

                        // Adem�s hacemos que ning�n plano pueda ser pickable

                        foreach (KeyValuePair<int, GeometryNode> pair in NodeRegister.planeNodes)
                        {
                            pair.Value.Physics.Pickable = false;
                            pair.Value.Enabled = false;
                            pair.Value.AddToPhysicsEngine = false;
                        }
                    }
                }
            }
        }

Jun 12, 2012 at 4:04 PM

Sorry, I tested again my function and dosnt work all the times :(

Jun 12, 2012 at 5:11 PM
Edited Jun 12, 2012 at 5:12 PM

Thank you for the code and the insight eagle17 :)

I've read the code and still trying to understand it,

would you mind to explain why did you put this code when you are trying to find the ray picked marker

 

findedList[i].IntersectParam < findedList[selectedMarker].IntersectParam

 

Is this code trying to find the object that is ray picked and intersected closest to the window?

and still don't understand what this code is actually do before you ray pick

 

 ((NewtonPhysics)sceneController.Scene.PhysicsEngine).RestartsSimulation();

 

currently I can't open the API documentation .chm for some reason so I cant find the meaning of .RestartsSimulations()

this workaround might work with my requirement too although it still need to be adapted and it doesn't modify any transformation in the physics engine except the deactivation and activation of physics, I haven't tested on my app though

Could you tell me the detail of "doesn't work all the times" as you have said?

 

Thanks :)

Jun 12, 2012 at 5:51 PM

Hello eng_front! Now I'm trying a new thing that maybe works. First, I only use one model for one marker, I don't know if this is your case...

 

Ok, I put the Propertie AddToPhysicsEngine of all of the GeometryNodes to false. 

When I'm doing the Picking, do this:


 geometryAssociated.AddToPhysicsEngine = true;




Matrix viewMatrix = MarkerController.Markers[i].WorldTransformation * GoblinXNA.State.ViewMatrix;

// Calculamos los puntos necesarios para el RayCasting
Vector3 nearPoint = sceneController.GraphicsDevice.Viewport.Unproject(nearSource, GoblinXNA.State.ProjectionMatrix, viewMatrix, Matrix.Identity);
Vector3 farPoint = sceneController.GraphicsDevice.Viewport.Unproject(farSource, GoblinXNA.State.ProjectionMatrix, viewMatrix, Matrix.Identity);

sceneController.Scene.Draw(new System.TimeSpan(), false);

                       

// Ahora calculamos el RayCasting y obtenemos los objetos interceptados
List<PickedObject> pickedObjects = ((NewtonPhysics)sceneController.Scene.PhysicsEngine).PickRayCast(nearPoint, farPoint);

                        
geometryAssociated.AddToPhysicsEngine = false;


What I'm doing is put the geometry Node property AddToPhysicsEngine to True, then I draw the scene (because drawing the scene, I'm updating the physics engine). Now I can do the RayCast with only the GeometryNode that I want presents in the physics engine.