LangChain Messages : Content vs ContentBlocks properties

OSS Product Help LangChain
1

In the LangCainJS documentation it is stated

Content blocks were introduced as a new property on messages in LangChain v1 to standardize content formats across providers while maintaining backward compatibility with existing code.

Content blocks are not a replacement for the content property, but rather a new property that can be used to access the content of a message in a standardized format.

I’m a bit confused by the phrasing here.

  1. Content blocks are not a replacement for the content property

When invoking a model with a HumanMessage, we can construct a HumanMessage with these contentBlocks.

Should we continue using the content property, or switch to the new contentBlocks property to define what we want to send ?

I believe all messages below are equivalent but content blocks offer a more standardized and type safe way of defining the input.

Or is the first option (using the standard langchain content format) sufficiently “standardized” to work across all providers already ?

// 1. Standard lanchain content format ?
new HumanMessage({
    content: [
        { type: "text", text: "Describe the content of this document." },
        { type: "image", source_type: "url", url: publicImageUrl, },
    ],
});

// 2. openai native content format ?
new HumanMessage({
        content: [
            { type: "text", text: "Describe this image." },
            {
                type: "image_url",
                image_url: { url: publicImageUrl },
            },
        ],
    });

// 3. new contentBlock typed api
new HumanMessage({
        contentBlocks: [
            {type: "text", text: "Describe this image."},
            {type: "image", url: publicImageUrl},
        ],
    });

Is using the new contentBlock types the way to go also when constructing messages (and if so would they not act as a replacement for the content property ?

  1. lazily parse the content attribute into a standard, type-safe representation.

Another use-case as shown in the docs seems to be to “lazy-load” the existing content property into the new contentBlocks typing.

Use-case here is having a type safe and unified way to parse model results (regardless of providers I assume ?) (at the time of writing not all providers have support for these contentBlocks so not all of them can be lazy loaded into the new typing.

Is my understanding correct here ?

2

Should we continue using the content property, or switch to the new contentBlocks property to define what we want to send ?

IMO yes! Once you get over the initial hurdle of how our content standard works, you can take a lot of the obscure cases that come with moving message objects between providers out of the equation.

(and if so would they not act as a replacement for the content property ?

We can’t represent every possible permutation of content that a provider will accept through a standard message abstraction (in other words, we couldn’t have a standard set of content types without adding some really obscure properties). We thought it would still be important to let you express those behaviors inside of content, but in order to also have a usable standard (and not break a bunch of existing user code) it had to be treated as separate.

Or is the first option (using the standard langchain content format) sufficiently “standardized” to work across all providers already ?

It is, but there are some notable feature gaps within the existing content format that we don’t standardize on across all providers. contentBlocks is meant to address that

(at the time of writing not all providers have support for these contentBlocks so not all of them can be lazy loaded into the new typing.

Important thing to clarify – all providers can be parsed into this standard contentBlocks, but when we say “it doesn’t have support” it more has to do with us not having verified that we’re representing every aspect of X provider’s message object as content blocks. If we don’t have a dedicated parser, we do a “best-effort” parse to get important content out (text + reasoning IIRC). Worst case is you drop a minimal amount of context when going from provider to provider