Tool calling in Lambda Inference API

post by dhruvmullick_aa on Jan 2, 2025

While using the Public Inference API, I get a blank output when using tools field, but legitimate output when not using tools in the data.

This is the payload I’m using:

{
        "model": "llama3.1-8b-instruct",
        "messages": [
          {
            "role": "system",
            "content": "You are a helpful assistant named Hermes, made by Nous Research."
          },
          {
            "role": "user",
            "content": "Who won the world series in 2020?"
          },
          {
            "role": "assistant",
            "content": "The Los Angeles Dodgers won the World Series in 2020."
          },
          {
            "role": "user",
            "content": "Where was it played?"
          }
        ],
        "tools": [
          {
            "type": "function",
            "function": {
                "name": "say",
                "description": "Say",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "message": {
                            "type": "string",
                            "description": "message"
                        }
                    },
                    "required": ["message"]
                }
            }
          }
        ]
      }

Output:

{"id":"","object":"","created":0,"model":"","choices":null,"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0,"prompt_tokens_details":null,"completion_tokens_details":null},"system_fingerprint":""}

post by cody_b on Jan 2, 2025

@dhruvmullick_aa I’m looking into this right now. I’ll post an update as soon as I have one.

post by cody_b on Jan 3, 2025

@dhruvmullick_aa The Lambda Inference API doesn’t support tool calling yet.

post by dhruvmullick_aa on Jan 3, 2025

I see. Thanks for getting back!

post by grantastic on Mar 29, 2025

Is there a timeline for this feature? I couldn’t find anything online. Unfortunately the lack of function calling is preventing us from using the service.

post by aswink on Apr 4, 2025

+1

Enabling tool calling would be super helpful in Agentic Use cases for me. ETA on this would be helpful.

post by cristiangofiar on Jun 24, 2025

Will you add the use of tools in the future? I need it now