Unity XR toolkit and Mirror networking, grabbing

I am new to both XR toolkit and Mirror and I have this problem that when I try to grab a box in online scene. It didn't work even thought it worked on the offline scene when I test it. anyone got any clue about this? or an example that I can use as a reference.

In my project I set the grabbable object to have a Network Identity , Network Transform and the xr grab interactable component, Then I set my player model(XR rig) to call server to assign authority when their hands touch the object. but for some reason there is no error or warning but I still can't grab/pick up the object

here's the code for authority assigning on the player hands:

   public class CubeAuth : NetworkBehaviour
{
    [SerializeField] NetworkIdentity me;
    private void OnTriggerEnter(Collider other)
    { 
        if (other.tag == "PickUp")
        {
            Debug.Log(other.gameObject.GetComponent<NetworkIdentity>().hasAuthority);
            CmdAssignNetworkAuthority(other.gameObject);
            Debug.Log("hit");
        }
    }
    [Server]
    //[Command(requiresAuthority = false)]
    public void CmdAssignNetworkAuthority(GameObject cube)
    {
        Debug.Log(cube.GetComponent<NetworkIdentity>().connectionToClient);
           /*if (cube.GetComponent<NetworkIdentity>().hasAuthority)
        {
            //Debug.Log("inside");
            // Remove authority
            cube.GetComponent<NetworkIdentity>().RemoveClientAuthority();
        }*/
        cube.GetComponent<NetworkIdentity>().AssignClientAuthority(me.connectionToClient);
        //cube.GetComponent<NetworkIdentity>().AssignClientAuthority(connectionToClient);

    }

p.s. it not an authority problem I tried.