Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement pick and place tasks + some improvements and fixes #289

Open
wants to merge 73 commits into
base: master
Choose a base branch
from
Open

Implement pick and place tasks + some improvements and fixes #289

wants to merge 73 commits into from

Commits on Apr 23, 2024

  1. try to visualize sokoban - may need nore neat way

    @lf-zhao
    lf-zhao committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    0824ae7 View commit details
    Browse the repository at this point in the history

  2. add assert for fast downward planner in running once func

    @lf-zhao
    lf-zhao committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    bf4000d View commit details
    Browse the repository at this point in the history

  3. try to use rich package for more structured console output!

    @lf-zhao
    lf-zhao committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    19d9d53 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. upload a naive way to store images

    @lf-zhao
    lf-zhao committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    feaa264 View commit details
    Browse the repository at this point in the history

  2. debug

    @lf-zhao
    lf-zhao committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    dd1fbf6 View commit details
    Browse the repository at this point in the history

  3. upload - manual copy from Nishanth's VLM interface in LIS predicators… 

    …; we need to figure out a better way to sync later!
    @lf-zhao
    lf-zhao committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    32a06a3 View commit details
    Browse the repository at this point in the history

  4. add OpenAI vlm - in progress

    @lf-zhao
    lf-zhao committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    ee3f634 View commit details
    Browse the repository at this point in the history

  5. update config setting for using vlm

    @lf-zhao
    lf-zhao committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    a1a67fd View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. add package

    @lf-zhao
    lf-zhao committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    5d9f12e View commit details
    Browse the repository at this point in the history

  2. another missed one

    @lf-zhao
    lf-zhao committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    cf3dbf7 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. manually add Nishanth's new pretrained model interface for now, see L… 

    …IS predicators!
    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    3001f32 View commit details
    Browse the repository at this point in the history

  2. add new OpenAI VLM class, add example to use

    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    4d47211 View commit details
    Browse the repository at this point in the history

  3. add a flag for caching

    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    e45a0e9 View commit details
    Browse the repository at this point in the history

  4. now the example working - fix requesting vision messages, update test

    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    8786dcd View commit details
    Browse the repository at this point in the history

  5. update; add choosing img detail quality

    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    71d0b3e View commit details
    Browse the repository at this point in the history

  6. include the test image I used for now, not sure what I should do with… 

    … this image
    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    112b690 View commit details
    Browse the repository at this point in the history

  7. remove original vlm interface, already merged into latest pretrained … 

    …py one
    @lf-zhao
    lf-zhao committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    f884df7 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Merge branch 'refs/heads/master' into lis-spot/implement-vlm-predicat… 

    …e-eval

# Conflicts:
#	predicators/pretrained_model_interface.py
#	setup.py
    @lf-zhao
    lf-zhao committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    8c17c42 View commit details
    Browse the repository at this point in the history

  2. found a way to use VLM to evaluate; add current images and also visib… 

    …le objects
    @lf-zhao
    lf-zhao committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    aec70de View commit details
    Browse the repository at this point in the history

  3. found a way to use VLM to evaluate; check if visible in current scene… 

    …, only update these predicates
    @lf-zhao
    lf-zhao committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    94e6a4c View commit details
    Browse the repository at this point in the history

  4. update State struct; adding to Spot specific subclass doesn't work, n… 

    …o idea why
    @lf-zhao
    lf-zhao committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    3ee2ba9 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. add detail option

    @lf-zhao
    lf-zhao committed May 1, 2024
    Configuration menu
    Copy the full SHA
    1abf488 View commit details
    Browse the repository at this point in the history

  2. working; implement On predicate with VLM classifier pipeline! add cal… 

    …l VLM and more
    @lf-zhao
    lf-zhao committed May 1, 2024
    Configuration menu
    Copy the full SHA
    1c82c44 View commit details
    Browse the repository at this point in the history

  3. make a separate function for vlm predicate classifier evaluation

    @lf-zhao
    lf-zhao committed May 1, 2024
    Configuration menu
    Copy the full SHA
    eeb1583 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. add test

    @lf-zhao
    lf-zhao committed May 3, 2024
    Configuration menu
    Copy the full SHA
    acbdb0a View commit details
    Browse the repository at this point in the history

  2. update example, move to test, move img

    @lf-zhao
    lf-zhao committed May 3, 2024
    Configuration menu
    Copy the full SHA
    01498f6 View commit details
    Browse the repository at this point in the history

  3. remove

    @lf-zhao
    lf-zhao committed May 3, 2024
    Configuration menu
    Copy the full SHA
    0774354 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2024

  1. format

    @lf-zhao
    lf-zhao committed May 4, 2024
    Configuration menu
    Copy the full SHA
    68ef57d View commit details
    Browse the repository at this point in the history

  2. update

    @lf-zhao
    lf-zhao committed May 4, 2024
    Configuration menu
    Copy the full SHA
    6560a94 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. batch VLM classifier working on Spot!! add field to State, add VLMPre… 

    …dicate and VLMGroundAtom
    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    1596f68 View commit details
    Browse the repository at this point in the history

  2. batch VLM classifier eval: add vlm predicates fields to observation

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    98ae2a9 View commit details
    Browse the repository at this point in the history

  3. batch VLM classifier eval: function on batch query and parse

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2683a6b View commit details
    Browse the repository at this point in the history

  4. batch VLM classifier eval: provide VLM predicates to object finding, … 

    …evaluate all VLM predicates on all objects to build init obs for env reset
    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b0b7568 View commit details
    Browse the repository at this point in the history

  5. batch VLM classifier eval: add VLM predicate fields to state+obs, bui… 

    …ld init obs that query all objects + all VLM predicates, update VLM ground atoms for visible objects, update VLM predicate classifiers
    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b4718f7 View commit details
    Browse the repository at this point in the history

  6. formatting

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    3ca9032 View commit details
    Browse the repository at this point in the history

  7. remove some comments

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    1ab3d36 View commit details
    Browse the repository at this point in the history

  8. remove some comments

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    11f91ff View commit details
    Browse the repository at this point in the history

  9. fix, add tenacity

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    c98e2fe View commit details
    Browse the repository at this point in the history

  10. fix structs

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    4973f1f View commit details
    Browse the repository at this point in the history

  11. more fix

    @lf-zhao
    lf-zhao committed May 8, 2024
    Configuration menu
    Copy the full SHA
    26239e8 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. update

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    e78e342 View commit details
    Browse the repository at this point in the history

  2. some clean

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    ce43a76 View commit details
    Browse the repository at this point in the history

  3. fix no VLM case

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    93fb574 View commit details
    Browse the repository at this point in the history

  4. add predicate prompt; fix and clean

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    cb0e1ee View commit details
    Browse the repository at this point in the history

  5. add predicate prompt & some logging; fix and clean

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    63a1429 View commit details
    Browse the repository at this point in the history

  6. overwrite vlm predicate classifier; reformat

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    ba2575f View commit details
    Browse the repository at this point in the history

  7. update

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    a11e5a1 View commit details
    Browse the repository at this point in the history

  8. update vlm query in obj finding

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    8ba4c5d View commit details
    Browse the repository at this point in the history

  9. add a simple pick place task - pick block and place into bowl

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    372c191 View commit details
    Browse the repository at this point in the history

  10. update

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    3019b97 View commit details
    Browse the repository at this point in the history

  11. update obj metadata

    @lf-zhao
    lf-zhao committed May 9, 2024
    Configuration menu
    Copy the full SHA
    f3e6a45 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. minor

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    7c82fde View commit details
    Browse the repository at this point in the history

  2. ?? change width/radius again, now okay

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    499f639 View commit details
    Browse the repository at this point in the history

  3. add prompt to vlm predicate

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    51bf569 View commit details
    Browse the repository at this point in the history

  4. minor

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    86ef996 View commit details
    Browse the repository at this point in the history

  5. minor

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    6db3fbe View commit details
    Browse the repository at this point in the history

  6. Merge branch 'refs/heads/lis-spot/update-vlm-predicate-pipeline' into… 

    … lis-spot/pick-place-task-and-improve

# Conflicts:
#	predicators/spot_utils/skills/spot_find_objects.py
    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    27f2698 View commit details
    Browse the repository at this point in the history

  7. update VLM atom update rule; add prompt to VLM predicate

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    d6daa39 View commit details
    Browse the repository at this point in the history

  8. add prompt field to VLM predicate / atom

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    7a4d20e View commit details
    Browse the repository at this point in the history

  9. reformat

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    ff770ce View commit details
    Browse the repository at this point in the history

  10. fix hash error - override hash func again

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    0175868 View commit details
    Browse the repository at this point in the history

  11. update atom print logic

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    083ac9f View commit details
    Browse the repository at this point in the history

  12. update print for predicate eval

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    9bd7949 View commit details
    Browse the repository at this point in the history

  13. got fix: pick and place working! add curr obs atoms to last state, re… 

    …turn all
    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    ad1f754 View commit details
    Browse the repository at this point in the history

  14. minor

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    0f0e8ef View commit details
    Browse the repository at this point in the history

  15. try update container dx value

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    5004878 View commit details
    Browse the repository at this point in the history

  16. minor

    @lf-zhao
    lf-zhao committed May 10, 2024
    Configuration menu
    Copy the full SHA
    8e0c4e0 View commit details
    Browse the repository at this point in the history

Commits on May 11, 2024

  1. rich table print util

    @lf-zhao
    lf-zhao committed May 11, 2024
    Configuration menu
    Copy the full SHA
    83484fb View commit details
    Browse the repository at this point in the history

  2. add table; to fix for object find!

    @lf-zhao
    lf-zhao committed May 11, 2024
    Configuration menu
    Copy the full SHA
    5898a1f View commit details
    Browse the repository at this point in the history

  3. add print rich table for VLM atoms

    @lf-zhao
    lf-zhao committed May 11, 2024
    Configuration menu
    Copy the full SHA
    5ebd449 View commit details
    Browse the repository at this point in the history

  4. formatting

    @lf-zhao
    lf-zhao committed May 11, 2024
    Configuration menu
    Copy the full SHA
    dc6ae7b View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. clean up previous legacy version of individual VLM classifier

    @lf-zhao
    lf-zhao committed May 18, 2024
    Configuration menu
    Copy the full SHA
    0e0d6cf View commit details
    Browse the repository at this point in the history

Commits on May 19, 2024

  1. minor formatting

    @lf-zhao
    lf-zhao committed May 19, 2024
    Configuration menu
    Copy the full SHA
    13f21ce View commit details
    Browse the repository at this point in the history